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The  Amy  Mathematics  Steering  Committee  initiated  the  present  series 
of  conference*  In  October  1955.  It  1*  the  intent  of  thl*  Committee  that 
these  Design  Conference*  afford  an  opportunity  to  statistical  design  spe¬ 
cialists  and  Army  research,  development  and  testing  personnel  to  get  to¬ 
gether  and  exchange  views  and  experiences  in  this  rapidly  growing  field. 

It  is  also  the  intent  that  through  invited  addresses  and  special  panel  dis¬ 
cussions  many  of  the  new  developments  in  the  theory  of  Statistical  Design 
and  Analysis  of  Experiments  be  brought  to  the  attention  of  Army  scientists; 
they  can  then  make  use  of  these  new  theories  to  help  solve  some  of  their 
complicated  design  problems. 

It  is  of  interest  to  note  that  the  host  of  this  Conference,  the  Cam s tic 
Research  Laboratories,  has  for  many  yaars  recognised  the  importance  of 
mathematical  statistics  and  aetlvaly  applied  the  methods  in  Army  research 
and  devalopment.  In  1926  Dr.  L.  S.  Dadarlck  dartvad  loan  unpublished 
manuscript  the  probability  distribution  of  the  sample  range.  In  1936 
General  (than  Captain)  Laatta  I.  Irnon  formed  within  the  Ballistic  Research 
Laboratories  a  group  essentially  coooamad  with  scientific  sen  ..ung.  At 
about  this  same  time,  be.  with  Colonel  H.  H.  Xornig,  published  a  paper 
entitled  "Tha  Propoeed  tystem  of  BurveiUenoe  of  Whr  Reserve  Ammunition.  ■ 
Among  the  many  ML  reports  in  the  field  of  statiettee  two  papers  by  R.  H. 

Kent,  one  issued  in  1939  on  "The  Mott  Eoooomioal  Sample  8iae"  end  the 
other  in  1940  on  "The  Estimation  of  the  Probable  Error  from  Successive  Dif¬ 
ferences"  ere  representative  of  early  work  and  served  to  stimulate  additional 
research  in  this  field.  Other  early  Ordnance  contributions  include  the 
Ordnance  Sampling  Inspection  Tables.  These  were  very  important  during 
World  war  n  and  ware  the  foramanar*  of  the  tables  standardised  by  the 
Departroe  of  Defence  as  Military  Standard  10SA.  Mathematical  Statistics 
has  been,  end  is.  playing  an  important  role  in  the  continuing  research,  devel¬ 
opment  end  testing  activities  at  the  Aberdeen  Proving  Ground,  and  the  above- 
mentioned  papers  ere  hut  a  few  of  the  many  contributions  that  have  been  made, 
and  are  being  made,  to  this  field  by  the  scientist!  at  the  Billiatic  Research 
Laboratories. 

The  five  invited  hour  addresses  at  the  Sixth  Design  Conference  were 
delivered  by  F.  J.  Anscombe,  W.  S.  Conner,  J.  R.  Duffett,  J.  E.  Jackson, 
and  W.  J.  Youden.  Residuals,  experimental  designs,  reliability,  and  muln- 
•  aviate  analysis  were  the  topics  discussed  by  the  first  four  of  these  sneakers 
W  J.  »■  uden,  the  banquet  speaker,  talked  on  "The  Enduring  Values'-.  A 
pane!  'usslon  on  "Common  Pitfalls  tn  the  Design  and  Analysi-  ot  Expnn- 
was  organized  and  chalrmanned  by  G.  E.  P.  Box.  The  members 
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his  panel  were  C.  Daniel,  J.  S.  Hunter,  W.  %  Youden,-and  M.  Zelen. 

In  addition  to  these  addressee,  ten  papers  were  presented  m  the  Clinical 
Sessions,  fifteen  in  the  Technical  Sessions,  and  eight  by  title.  Special¬ 
ists  in  the  Clinical  Sessions  were  asked  to  discuss  experimental  designs 
in  the  areas  of  tolerance  and  calibration  problems,  optics,  bomb  salvos, 
fatigue  limits,  missile  safety,  and  multivariate  analysis.  In  the  Technical 
Sessions,  personnel  management,  simulation,  trajectory  analysis,  aerosol 
chamber  data,  nerve  gas  experiments,  reliability  of  weapon  systems,  and 
response  surface  analysis  were  but  a  few  of  the  topics  that  were  con- 
sldered. 

The  Sixth  Conference  was  attended  by  115  registrants  and  p.aici- 
pants  from  58  organisations  outside  of  the  Ballistic  Research  Labora¬ 
tories.  In  addition,  71  tuff  members  and  other  personnel  of  the  host 
organisation  were  present.  Speakers  end  panelists  oame  from  Bcoz, 

Allen  Applied  Research,  Inc. ,  Canadian  Army  Operational  Research 
Establishment,  Cornell  University,  Defence  Research  IfedHoal  Labora¬ 
tories  (Canada),  Eastman  Kodak  Company,  General  Analysis  Corporation, 
Hercules  Powder  Company,  National  Bureau  of  Standards,  Princeton 
University,  Research  Triangle  Institute,  Space  Technology  Laboratories. 
Inc. ,  University  of  California,  University  of  Chicago,  University  of 
Georgia,  University  of  Maryland,  University  of  Michigan,  University  of 
Wisconsin,  Virginia  Polytechnic  Institute,  and  15  Army  facilities. 

The  members  of  the  Army  Matheamtioe  Steering  Committee  take  this 
opportunity  to  express  their  thank*  to  the  many  speakers  end  other  re¬ 
search  workers  who  participated  in  the  Conference;  to  Brigs dier  General 
John  H.  Weber,  the  Commending  General  of  the  Aberdeen  Brovina  Ground, 
and  Colonel  J.  P.  He  ml  11,  Director  of  the  Ballistic  Research  Laboratories, 
few  making  such  excellent  facilities  available  for  the  Conference;  end 
to  Dr.  Prank  E.  Grubbs  who  served  as  Chairmen  on  Local  Arrangements . 
Thanks  are  due  many  others  at  the  Laboratories  for  the  time  end  the 
help  they  gave  the  .participants.  Of  these,  Mr.  O.  P.  Bruno  end  Major 
Joseph  E.  Sows  deserve  special  mention.  They  handled  many  of  the 
local  details  for  Dr.  Grubbs  and  organised  the  Interesting  tour  of  the 
local  facilities. 
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Finally,  the  Chalxmm.  wishes  to  express  Ms  appreciation  to  the  Ad¬ 
visory  Committee;  G.  E.  P.  Box,  F.  C.  Dressel  (Secretary),  Frank  E. 
Grubbs,  Boyd  Hershberger,  Clifford  J.  Maloney,  J.  S.  Hunter,  and' 
Marvin  Solen  fcr  their  help  in  organizing  the  program  of  the  Conference, 
and  especially  to  Dr.  Dressel  for  coordinating-  the  Conference  program 
and  steering  these  Proceedings  through  publication. 


S.  S.  Wilke 

Professor  of  Mathematics 
Princeton  University 


PROGRAM 


SIXTH  CONFERENCE  OH  TSSLDE31GN  OT  EXPERIMENTS  IN  RJWfflT  . 
RESEARCH,  DEVEXOPWEEKT  AND  TESOTSG 

19  -  21  October  i960 

Wednesday,  19  October 

REGISTRATION:  0845  -  0945  t£s stem  Daylight  Saving  Time) 

Theater  No.  1,  Aberdeen  Proving  Ground 
GENERAL  .SESSION  I:  0945  -1215  -  Theater  No.  1  . 

Galling  of  Conference  to  Order:  . 

Dr.  F.  E.  Grubbs,  Local  Chairman 
Welcome: 

Brigadier  General  J.  H.  Weber,  Commanding  General,  Aberdeen 
Proving  Ground 

Introduction: 

Colonel  J.  P.  HamlU,  Director  of  the  Ballistic  Research  Laboratories 
Chairman: 

Professor  S.  S.  Wilks,  Princeton  University 
Invited  Papers: 

Reliability 

Dr.  James  R.  Duffett,  Space  Technology  Laboratories,  Inc. 
Examination  of  Residuals 

Professor  F.  J.  Aracombe,  Princeton  University 
At  1215  buses  leave  Theater  No.  1  for  the  Chesapeake 


LUNCH:  1230  -  MOO  -  Ctoesaipmaka 

Wednesday  Afternoon 

There  will  be  three  Technical  Sessions  and  one  Clinical  Sesaloa  con¬ 
ducted  Wednesday  afternoon.  Technical  Sessions  1  and  U  will  he  held 
concurrently  from  MOO  to  1500.  From  1515  to  1645  Technical  Session  m  and 
Clinical  Session  A  will  be  running  concurrently.  The  security  classification 
of  the  first  paper  in  Technical  Session  III  Is  SECRET.  No  clearances  are 
required  for  the  other  papers  given  on  Wednesday. 

TECHNICAL  SESSION  1:  MOO  -  1500.  -  Chesapeake  -  Room  A 

.  Chairman:  Joseph  Weinstein,  U,  Sf.  Army  Signal' Research  and  . 
Development  Laboratory 

A  Simulation  Error-Model  for  an  Airborne  Target  Location  System  - 
E.  Blser  and  John  Beckmann,  U.  S.  Army  Signal  Research  and 
Development  Laboratory 

Analysis  of  Some  Trajectory  Measuring  Instrumentation  Systems  - 
O.  L.  Kingsley,  .Range  Instrumentation  Division,  White  Sands 
Missile  Range 

TECHNICAL  SESSION  II:  MOO  -  1500  -  Chesapeake  -  Room  B 

Chairman:  Clifford  J.  Maloney,  U,  S.  Army  Biological  Warfare 
Laboratories 

A  Trial  Comparing  Certain  Side  Effects  of  Two  Nerve  Gas  Antldcs.es. 
Using  Human  Subjects  -  C.  A.  de  Candole,  Defence  Research 
Medical  Laboratories,  Downsvlew,  Ontario,  and  B.  A.  Richardson, 
Canadian  Army  Operational  Research  Establishment,  Ottawa,  Canada 

An  Application  of  the  Exponential  Hazard  Model  to  Aerosol  Chamber 
Trial  Data  -  Theodore  W.  Horner,  Booz,  Allen  Applied  Research,  Inc. 


COFFEE:  1500  -  1515  -  Chesapeake 
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GUmCtiL  SESSION  At  1515  -  16'45  -dhesapesitae'-  Boom  4 

Chairman:  Ralph  E.  Brown,  Frankford  Arsenal 

Panelists:  G.E.P.  Box,  The  University  of  Wisconsin 

A.  C.  Cohen,  Jr.,  The  University  of  Georgia 

W.  S.  Connor,  The  Research  Triangle  Institute 

K.  A.  David,  Virginia  Polytechnic!  Institute 

J.  R.  Duffett,  Space  Technology  Laboratories,  Inc. 

Calibration  of  a  Zinc  Sulfide  Particle  Detector  •*  John  E.  Malllgo. 
Methods  Research  Section,  MR  &  AE  Branch,  Technical  Evaluation 
Division,  Chemical  Corps  Biological  Laboratories 

Effects  'of  Aiming  Point  Patterns  on  Bomb  Salvo  Target  Coverage  - 
R.  D.  Doner,  Systems  Analysis  Laboratory,  OML  Division,  Arnqr 
Rocket  and  Guided  Missile  Agency,  Redstone  Arsenal 

The  Tolerance  Structure  of  Complex  Systems  -  William  S.  Agee. 
Flight  Simulation  Laboratory,  White  Sands  Missile  Range 

TECHNICAL  SESSION  HI:  1515  -  1645 

The  first  paper  In  this  session  carries s  a  security  clesalflcatlon  of 
SECRET  and  will  be  held  In  Room  259,  BRL  Bldg.  328.  Transportation  from 
the  Chesapeake  to  BRL  Bldg.  328  will  be  provided  at  1505  hours. 

The  second  paper  will  be  given  in  Room  B,  the  Chesapeake,  beginning 
at  1610  hours.  Transportation  from  BRL  Bldg.  328  back  to  the  Chesapeake 
will  be  provided  at  1600  hours. 

Chairman:  F.  Howard  Forsyth,  Office,  Chief  of  Ordnance,  Department 
of  the  Army 


Allocation  of  Resources  and  Military  Worth  -  Walter  E.  C  us  hen. 
Operations  Research  Office,  The  Johns  Hopkins  University 


vtii 


lEgmasMt.  «s8iQOUggiK,,<a 

Room  B,  Chesapeake 

An  Experiment  in  Personnel  Management  Evaluation  -  Richard  R. 
Blough,  Statistical  Research  Center,  The  Univeraity  of  Chicago 

Wednesday  Evening?  The  cocktail  lounges  at  the  Chesapeake  and  th'1 
Main  Club  ere  open  from  1630  to  2300  hours.  The  dining  room  at  the  Mam 
Club  is  open  from  1800  to  2000  hours. 

Buses  will  take  conferees  to  motels  or  Main  Club. 

ThM£l4iYAl.^Q . flgfifell 

Clinical  Session  B  carries  a  security  classification  of  SECRET.  It  and 
Technical  Session  W  will  run  from  0900  to  1015.  Clinical  Session  C  and 
Technical  Session  V  scheduled  from  103-i  to  1230  complete  the  morning  phase 
of  the  program.  In  the  axtemoon  Technical  Sessions  VI  and  VII  run  concur¬ 
rently  from  1400  to  1440.  General  8etsion  II  will  be  a  panel  discussion 
and  is  timed  from  1500  to  1645. 

TECHNICAL  SESSION  IV:  0900  -  1015  -  Chesapeake  -  Room  A 

Chairman:  Gertrude  Welntraub,  Missile  Warhead  and  Special  Projects 
Laboratory,  Pica  tinny  Arsenal 

Reliability  of  Weapon  Systams  Estimated  from  Component  Test  Data 
Alone  -  Henry  DeCtcco,  U.  8.  Army  Ordnance  Special  Weepons- 
Ammunitlon  Command 

Performance  of  Propellants  Evaluated  by  Tansila  and  Ballistic  Tests  - 
Niles  White,  Propellant  Branch,  Propellant  Laboratory,  ARGMA,  and 
Boyd  Harshbarger,  Virginia  Polytechnic  Institute 

CLINICAL  SESSION  B;  0900  -  1015  -  BRL  Bldg.  328,  Room  259 

security  Classification  -  SECRET 

"5 n porta t ion  from  the  Chesapeake  to  BRL  Bidg.  328  will  be  available 
'^4-  hours.  Bus  from  BRL  Bldg.  328  to  Chesapeake  at  1015  hours) 
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Chairman:  Edward  W.  Chittenden,  Diamond  Ordnance  Fuze  Laboratories 

Panelists:  R,  M.  ElsSnsr,  Ballistic  Research  Laboratories 

Walter  Foster,  U.  S.  Army  Biological  Warfare  Laborr 
atones 

J.  R.  Johnson,  Ballistic  Research  Laboratories 

Clifford!.  Maloney,  U.  S.  Army  Biological  Warfare 
Laboratories 

S.  S.  Wilks,  Princeton  University 

Marvin  Zelen,  University  of  Maryland 
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sions  of  Variables  Requiring  Transformations  -  Jerzy  Neyman  and 
Elizabeth  L.  Scott,  Statistical  Laboratory,  University  of  California, 
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James  R.  Duffel? 

Space  Tffcbnoloqy  Laboratartfta,,  Ttku 


A.  BACKGROUND.  These  personnel  who  have  been  involved  in  the  flight 
testing  of  complex  guided  missiles  ere,  in  general,  aware  of  t'ne  phenomenon 
that  a  much  higher  system  reliability  (R). is  obtained  for  an  essentially  aeries  " 
arrangement  of  the  components  than  would  prevail  if  estimates  of  the  compo¬ 
nent  reliabilities  ir^'s,  1*1,  2,  ....  n)  were  substituted  into  the  mathemati¬ 
cal  model 

R-rjTj  ...rn. 

This  phenomenon  hat  been  eloquently  exposed,  and  elucidated  upon,  by 
Frank  A.  Fleck  of  United  Electro  Dynamics.  The  model 
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R  *  r,  r.  ...  r 

■»  z  n 

is  usually  referred  to  as  the  "senes'*  model;  ttiialsoknownei  the  "cascade" 
or  "tandem"  model.  By  contrast,  those  personnel  who  have  been  concerned 
with  the  use  of  redundancy  (l.e.,  the  paralleling  of  components)  in  order  to 
lower  th’i  probability  of  a  dud  (lie.,  the  unreliability  of  the  payload)  have 
observed  a  phenomenon  which  is  opposite  in  effect  to  that  which  hae  bean 
observed  in  the  series  case.  Namely,  a  system  of  components  arranged  in 
parallel  evidences,  in  general,  a  lower  reliability  (R)  than  would  be  obtained 
if  the  estimates  of  the  component  reliabilities  (r^'s,  i  -  1,  2,  m)  were 
substituted  into  the  mathematical  modal 


R  -  l  -  0  -  r,)U  -  r2)  ...  U-rm). 


This  last  model  Is  usually  referred  to  as  the  "redundant"  or"parallel"  model. 

A  plausible  explanation  for  these  two  phenomena,  which  heve  op¬ 
posite  effects  for  the  series  and  redundant  systems,  is  that  the  variation  of 
the  environmental  stresses  from  component  to  component  within  the  same 
flight  is  small  relative  to  the  variation  of  the  environmental  stresses  from 
flight  to  flight.  It  is  believed  that,  in  many  redundant  systems,  the  stresses 
associated  with  the  m  parallel  components  within  the  seme  flight  possess 
such  a  small  standard  deviation  that  this  standard  deviation  can  be  neglected; 
in  such  instances  the  variation  attributed  to  the  stresses  is  essentially  the 


"This  is  an  abstract.  The  paper  Itself  is  being  submitted  for  publication  In 
TECHNOMETRICS. 
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ntftTtdorti  ra£  tfia  «xaaa««  tfcom ‘flight  to  flight.  This  belief  la  attri¬ 

buted  to  the  fallowing  facts: 

(1)  The  m  parallel  components  are  usually  mounted  in  dose 
physical  proximity  to  each  other  and  hence  tend  to  heve 
the  same  quantitative  values  of  the  stresses. 

(2)  the  in  parallel  components  are  usually  of  the  same  type 
and  thus  are  subject  to  the  same  mode  of  failure  and  hence 
are  susceptible  to  tie  same  hind  of  stresses. 

(3)  Many  kinds  of  stresses  vary  considerably  from  flight 
to  flight. 

These  same  facts  can  prevail  in  the  case  of  some  aeries  systems,  tig. ,  the 
successive  amplifier  stages  In  en  equipment,  the  electron  tubes  in  a  guid¬ 
ance  end  control  package,  and  the  relays  In  a  black  box. 

W.  ].  Howard (1)  has  considered  the  series  situation  for  a  specific  num¬ 
erical  value,  of  the  component  reliability  r  end  for  Gauselen  distributions  of 
strengths  and  flight -to~fllght stresses,  whereas  7.  R.Duffett^)  has  considered 
the  parallel  situation  for  a  large  range  of  values  of  the  component  reliability 
r  and  rectangular  distributions  of  strengths  and  flight -to-f light  stresses  plus 
certain  generalizations  of  these  assumptions , 

An  Interesting  pathological  example  of  the  complete  breakdown  of  redun¬ 
dancy  la  afforded  by  the  following  example  which  is  presented  in  (2): 

The  m  parallel  components  incorporated  in  the  same  missile 
flight  (l.e.,  the  same  end  Item)  are  subjected  to  exactly  the  same 
stress;  tho  probability  distribution  of  the  stresses  from  fllght-to* 
flight  consists  of  two  isolated  portions;  and  the  probability  distri¬ 
bution  of  strengths  is  sandwiched  In  between  the  two  Isolated 
portions  of  the  stress  distribution. 

It  Is  clear  that  the  reliability  R  of  the  system  la  only  r,  l.e.. 

R  ■  r,  and  thus  no  gain  In  reliability  is  achieved  by  using  redundancy. 


(1)  W,  I.  Howard,  "Chain  Reliability:  A  Simple  Failure  Model  for  Complex 
Mechanisms,"  The  Rand  Corporation,  RM-1058,  27  M«;  jh  H53. 

(2)  7.  R.  Duffett,  "Some  Mathematical  Considerations  of  Redundancy" , 
Radioplane  Company,  Operations  Analysis  Memorandum  Report  Number 
12,  25  October  19SS. 
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Or,  such*  situation,  as  that  da  Berthed;  inthe  foregoing  example,  the 
saliabflttf*  S'  ef  a  sgafcsnv  would;  be  r,  Co.,  R«r,  regattdlisat  of  the 
design  asraorjeaferti  v&  tine  component*  -which  compose  the  system. 

An  extremely  pathological  example  which  illustrates  a  situation  lit 
which  the  opposite  effect  (as  that  discussed  above)  Is  obtained  has  been 
given  by  C.  R.  Gates  and  Is  stated  as  follows: 

A  system  Is  composed  of  2  components.  One  component 
falls  If  and  only  If  the  temperature  Is  greater  than  or  equal  0° 

F,  whereas  the  other  component  fails  If  and  only  If  the  tempera¬ 
ture  Is  less  than  0°  F.  If  the  components  are  arranged  In  series, 
the  reliability  of  the  system  will  be  zero.  However,  if  the  com¬ 
ponents  are  arranged  in  parallel,  the  reliability  of  the  system 
will  be  one. 

7n  order  to  simultaneously  control,  at  acceptable  levels,  both  the 
probability  of  a  dud  (or  a  late  detonation)  and  the  probability  of  a  "pre¬ 
mature",  some  technical  personnel  have  proffered,  as  a  desirable  solu¬ 
tion,  tha  use  of  a  "matrix"  of  components,  arranged  both  in  parallel  and 
in  oeriesjl.e, ,  the  system  is  to  consist  of  m  paallel  circuits  (or  branches), 
where  each  circuit  has  n  components  arranged  in  series(3. 4).  Such  a 
system  will  be  referred  to  as  a  parallel-series  system.  (Apparently  for 
analytical  simplification,  m  and  n  ace,  in  general,  set  equal  to  each 
other  so  that  the  design  matrix  is  square.)  A  special  case  of  a  parallel- 
series  system  is  the  quad.  The  quad  is  of  considerable  engineering  interest 
and  consists  of  2  parallel  circuits,  each  possessing  2  components  arranged 
in  series. 

Another  design  arrangement,  which  tends  to  decrease  the  percentage 
of  duds  while  increasing  the  percentage  of  prematures,  is  a  series-paral¬ 
lel  system.  This  arrangement  consists  of  n  circuits  (or  links)  in  series, 
where  each  circuit  has  m  components  In  parallel. 

B.  SYSTEMS  STUDIED  AND  ASSUMPTIONS  MADE.  In  this  document,  the 
reliability  of  the  four  types  of  systems— (1)  a  simple  parallel  system,  (2) 
a  simple  series  system,  (3)  a  parallel-series  system,  and  (4)  a  series-  ' 
parallel  system — are  derived  under  the  following  assumptlonsl 


(3)  Burton,  £.  B. ,  The  Martin  Company.  Unpublished  paper  on  redundancy 
quads,  and  crossing  circuitry. 

(4)  Crcvcllng,  C.  I.,  "Increasing  the  Reliability  of  Electronic  Equipment 
by  Use  of  Redundant  Circuits,"  NRL  Report  4631,  5  December  1955, 
Naval  Research  Laboratory,  Washington,  D.  C. 
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fc)}  Tata'  'firenqtits.  ofithtt  campanewts;  regardless  at  the  flight 
a*  wfttch  they  are  Incorporated,  are  Independently  selected 
Swam  a  ftaesd  recta  rosuAar  distribution.  - 

(b)  All  components  which  are  incorporated  tnfria  same  flight 
experience  exactly  the  same  stress, 

(c)  The  common  stress,  which  is  applied  to  all  of  the  compo¬ 
nents  within  the  same  flight,  is  independently  selected  from 
a  fixed  rectangular  distribution. 

(d)  Failure  of  a  component  .occurs  if,  and  only  if,  the  stress 
Imposed  on  It  exceeds  ltc  strength. 

(e)  Failure  of  a  simple  series  arrangement  of  the  components 
occurs  If,  and  only  if,  at  least  one  of  the  components  fells. 

(f)  Failure  of  a  simple  parallel  arrangement  of  the  components 
occurs  If,  and  only  if,  all  of  the  components  fail, 

(g)  Failure  of  a  parallel -aeries  systevn  occurs  if,  and  only  if, 
all  of  the  parallel  branches  (circuits)  fall. 

(h)  Failure  of  a  series -parallel  system  occurs  if,  and  only  if, 
at  least  one  of  the  links  (circuits,  in  series-  fails. 

1.  CONCLUSIONS. 


(1)  It  Is  concluded  that  the  reliability  of  a  System  composed  of  m  paralle, 
components  can  approach,  as  an  upper  limit,  the  value  given  by  the 
Independence  model,  viz. , 

R  -  I  -  0  -  r2)0  -  r2)  . . ,  0  -  rj, 

as  the  variation  of  the  stresses  between  flights  decreases  relative  to 
the  variation  of  the  component  strengths. 

(2)  It  is  concluded  that  the  reliability  of  a  system  composed  oi  n  series 
components  can  approach,  as  a  lower  limit,  the  value  given  by  the 
independence  model,  viz.. 
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tha  vartatt’on  oCthe  stresses  between  flights  decmtts  telaftSvm 
to  toe  -vaiatSoa.  of  the  component  strengths . 

It-  Is  further  concluded  that,  for  a  given  level  of  component  reliability, 
the  reliability  of  a  series  system  can  be  Increased  by  decreasingthe 
copy-to-copy  variation  of  the  strengths  of  the  components, 

(3)  It  Is  concluded  drat  the  reliability  of  a  system  composed  of-  m  para¬ 
llel  components  can  approach,  as  an  upper  limit,  the  value  given  by 
the  independence  model,  vis., 

0-rj, 

as  the  variation  of  the  stresses  between  flight*  decreases  relative  to 
the  variation  of  the  stresses  within  flights. 

(4)  It  Is  concluded  that  the  reliability  of  a  system  composed  of  n  series 

components  can  approach,  as  a  lower  limit,  the  value  given  by  the 
Independence  model,  viz. ,  - 

*'rlr2**‘V 

as  the  variation  of  the  stresses  between  flights  decreases  relatlveto 
the  variation  of  the  stresses  within  flights. 

(5)  It  is  concluded  that  one  may  obtain  unwarranted  optimistic  e  stimates 
of  the  reliability  of  a  system  composed  of  parallel  components  If  the 
Independence  model  is  assumed,  but  not  satisfied. 

(6)  It  is  concluded  that  one  may  obtain  unwairanted  pessimistic  estimates 
of  the  reliability  of  a  system  composed  of  series  component.1  If  the 
Independence  model  is  assumed,  but  not  satisfied. 

M.  RECOMMENDATIONS. 

(D  It  is  recommended  that  systems  Integration  studies  be  made  for  the 
purpose  of  determining  the  effect  of  component  failures  on  system 
effectiveness  and  consequently  to  classify  the  systems  Incorporated 
in  the  end  item  according  to  such  categories  as  series,  parallel, 
series -parallel,  and  parallel-series. 

(2)  If  parallel  components  are  to  be  used,  then  It  is  recommended  that 
consideration  be  given  to  the  following: 
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(a)  the  selection  of  components  whose  modes  of  failure  ere  nBMhneutL. 

.  (b)  The  incorporation  of  the  components  in  such  a  way  that  the  stresses 
associated  with  the  components  are  Independently  selected  from  e 
fixed  probability  distribution  regardless  of  the  end  Item  In  which 
the  component  is  incorporated.. 

It  may  be  possible  to  Implement  tb)  by  physically  separating  the 
parallel  components  by  sufficiently  large  distances,  or  otherwise 
isolating  the  perellel  components  from  one  another. 

(c)  The  isolation  of  the  parallel  components  from  their  environments. 

(3)  If  series  components  ere  to  be  used,  then  it  is  recommended  that  con¬ 
sideration  be  given  to  the  following: 

(a)  The  selection  of  components  with  (approximately  )  the  tame  modas 
of  failure. 

(b)  The  assembly  of  the  components  in  such  a  manner  that  they  will 
experience  (approximately)  the  same  environmental  regime. 

One  method  with  which  to  Implement  (b)  is  to  package  the  series 
components  as  a  compact  unit. 

(4)  It  is  recommended  that  the  existing  relevant  transportation  and  in-flight 
envlrqnments  which  have  been  measured  be  evaluated,  It  Is  further 
recommended  that  consideration  be  given  to  the  instrumentation  of 
missiles  for  the  purpose  of  obtaining  additional  Information  on  the  en¬ 
vironmental  conditions  which  ate  encountered  by  missile  systems  both 
(1)  within  flights  and  (U)  from  flight  to  flight. 

(5)  It  is  recommended  that  the  probability  distributions  of  component 
strengths  b»  determined,  primarily  by  means  of  laboratory  tests.  It  Is 
further  recommended  that  consideration  be  given  to  decreasing  the 
variability  of  the  strengths  of  components  which  ere  to  be  used  In 
series  systems  and  subsequently  to  maintaining  (through  Statistical 
Quality  Control)  the  variation  of  component  strengths  at  a  satisfactory 
level. 

(6)  It  Is  recommended  that  the  formulas  given  in  this  paper  be  employed 
to  serve  as  a  guide  in  the  calculation  of  system  reliability. 
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During  the  lest  few  years  there  has  been  a  growing  Interest  In  exam¬ 
ining  the  residuals  after  dome  parameters  have  been  fitted  by  the  method  of 
least  squares.  The  author  first  became  Interested  In  the  subject  in  1954 
through  some  suggestions  made  by  John  W.  Tukey  [Q  :  and  he  has  recently  . 
been  concerned  In  several  studies  relating  to  residuals  [2,3.4] .  The  pur¬ 
pose  of  this  paper  is  to  make  a  brief  introductory  sketch  of  some  of  these 
developments.  The  presentation  will  be  in  terms  of  a  particular  example. 

In  Section  1  an  experiment  is  described  and  the  conventional  type  of  statis¬ 
tical  analysis  Is  outlined.  In  Section  2  comments  are  made  on  the  validity  o( 
the  analysis.  In  Section  3  the  residuals  associated  with  the  conventional 
analysis  are  examined,  and  suggestions  are  made  for  modifying  the  analysis. 

I.  A  LATIN -SQUARE  EXPERIMENT  AND  ITS  STANDARD  ANALYSIS.  In  Table  1 
are  shown  a  set  of  observations  of  depth  of  penetration  of  a  bleat  driven 
earth  rod.  Ten  different  propelling  charge  lots  (denoted  by  the  letters  A,B, 

C, . . . ,  J)  were  compared  on  ten  different  sites  or  “plats'*  (shown  as  coh>  iras 
.In  the  tablej,  firings  being  made  on  ten  different  dates  spread  over  a  period 
of  some  months  (the  dates  are  termed  "blocks*  and  shown  as  rows  in  the  table). 
In  accordance  with  a  10  x  10  Latin  square  pattern.  In  each  cell  of  the  Latin 
square  (that  la,  at  each  date,  on  each  plot)  the  appropriate  propellant  lot  was 
tested  in  duplicate,  end  two  holes  were  driven.  Thus  there  were  200  read¬ 
ings  In  all.  (These  data  have  been  kindly  supplied  by  Dr.  Frank  E.  Grubbs.) 


For  such  a  set  of  readings,  arranged  In  a  Latin  square  design,  then 
is  a  standard  method  of  statistical  analysis,  which  a  statistician  Is  likely 
to  follow  almost  without  thinking.  The  sum  and  the  difference  of  the  pair  of 
entries  in  every  cell  of  the  Latin  square  are  calculated.  The  sum  of  squares 
of  the  differences  Is  found,  to  obtain  a  wlthln-cell  estimate  of  error  variance; 
the  Individual  differences  are  then  forgotten  about.  From  the  sums  of  pairs 
of  cell  entries,  row,  column  and  letter  means  are  calculated.  If  we  denote 


the  sum  of  the  two  readings  In  the  cell  In  the  l_th  row  and  Jth  column  by  y^j, 
then  the  various  means  to  be  calculated  are  the  row  means  ,  the  column  mean 
y.j.  the  overall  mean  y,  and  the  letter  means,  which  may  be  denoted  by  y/*»,...,  y«*. 
These  row,  column  and  letter  means  show,  respectively,  the  effects  ofblocksW 
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(dates),  plots  and  prppeilSwre  tots*  and  can  be  set  out  in  {fence  shot  Abbes, 
such  as  Tahle  2,  relating  fir®  tprajpeUlant  lots.  The  entries  In  this  table  are  ttn 
fact  the  letter  means  divided  bv  2.  so  that  we  have  average  penetrations  per 
firing,  rather  than  averages  of  sums  of  two  penetrations. 

We  can  obtain  an  overall  picture  of  the  amount  of  variation  present 
in  the  readings  by  constructing  the  enalysls  of  variance  table  shown  in  Table 
3.  For  the  purpose  of  comparing  means  of  rows  or  of  columns  or  of  letters, 
one  would  take  the  residual  mean  square  In  the  analysis  of  sums  of  cell  pairs 
namely  63.6,  as  the  estimated  residual  variance.  The  estimated  standard 
error  shown  in  Table  2  Is  equal  to  one -half  (because  the  letter  means  were 
divided  by  2)  of  >f?S767Io’.  It  will  be  seen  that  there  Is  no  evidence  that 
the  propelling  charge  lots  have  any  differential  effect  on  depth  of  penetration. 
There  is  a  marked  seasonal  effect,  end  there  seems  to  be  a  plot  effect  also. 

II.  COMMENTS  ON  THE  STANDARD  ANALYSIS.  The  customary  analysis  of 
experimental  data  goes  along  the  lines  briefly  Indicated  above.  Is  it  satis¬ 
factory?  The  orthodox  treatment  of  the  data  would  be  perfectly  appropriate  ■ 
and  valid  if  certain  Ideal  conditions  were  satisfied.  We  have  no  reason  t< 
suppose  that  these  conditions  ore  ever  satisfied  exactly,  but  it  may  well  be 
that  they  are  often  nearly  enough  satisfied  for  practical  purposes-.  The  ideal 
conditions  are  sometimes  referred  to  as  the  assumptions  underlying  the  analy¬ 
sis  of  variance.  For  the  present  T;aUn  square  design,  they  are  as  follows: 

IDEAL  CONDITIONS.  The  observstlons  are  realizations  of 
Independent  chance  variables  all  normally  distributed  with 
the  same  variance  and  with  moans  consisting  of  a  tow  con¬ 
stant  plus  a  column  constant  plus  a  letter  constant. 

Was  it  reasonable  to  analyze  the  observations  as  though  these  con¬ 
ditions  were  satisfied?  One  may  question  the  standard  statistical  analysis 
of  any  body  of  data  under  the  three  main  headings: 

(1)  Are  the  observations  trustworthy?  Should  they  be  taken  at  face 
value? 

If  the  answer  la  yes,  * 

(2)  Are  the  Ideal  conditions  nearly  enough  satisfied  to  make  the 
standard  analysis  acceptable? 

If  the  answer  is  no,  or  doubtful. 
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(3)  .  Hyw  should  the  Standard  analysts  be  modified ior  rcplacad? 

In  regard  to  fl),  no  observations  ere  absolutely  trustworthy.  If  the 
results  ere  sufficiently  at  variance  with  expectation,  a  mistake  In  the  ob¬ 
servations  will  be  strongly  suspected.  Sometimes  it  will  be  possible  to 
verify  directly  that  a  mistake  has  occurred,  and  perhaps  to  rectify  It.  But 
even  if  It  la  not  possible  to  repeat  or  check  the  observations,  a  verdict  of 
"presumed  mistake"  may  still  seem  the  most  reasonable,  and  that  implies  ' 
that  observations  will  be  discarded.  In  some  cases  the  whole  of  the  ob¬ 
servation  may  be  discarded.  In  other  cases  just  one  or  two  aberrant  read¬ 
ings  may  be  picked  out  as  presumably  spurious,  the  rest  being  accepted  as 
reliable. 

In  regard  to  (2)  ,  the  divert  ways  in  which  the  Ideal  conditions  could 
fall  to  be  satisfied  are  unlimited  in  number.  The  means  may  fall  . to  have  the 
specified  simple  linear  structure ,  and  the  deviations  of  the  observations  from 
the  means  could  in  principle  have  any  stochastic  character  whatever.  There 
are,  however,  a  few  types  of  departure  from  the  Ideal  conditions  that  seem 
to  be  worth  looking  for  explicityiy,  as  being  easily  Intclllblble  and  possibly 
important. 

Oft  the  subject  of  how  far  various  kinds  of  departure  from  the  ideal 
conditions  invalidate  the  standard  method  of  analysis,  not  as  much  Is  known 
as  one  might  wish.  (This  topic  is  reviewed  in  the  last  chapter  of  XlQ  and 
in  chapter  5  of  [6]  .)  If  the  Ideal  conditions  were  exactly  satisfied,  the 
standard  analysis  would  be  the  most  convenient  and  intelligible  end  efficient 
possible.  In  so  far  os  the  ideal  conditions  are  not  satisfied,  the  standard 
analysis  will  be  In  some  degree  Inappropriate  and  perhaps  misleading.  For 
large  enough  departures  from  the  Ideal  conditions,  it  would  be  preferable  to 
perform  some  sort  of  modified  or  alternative  analysis,  but  that  means  further 
computation  and  possibly  less  easily  intelligible  results. 

Examination  of  residuals  Is  a  valuable  method  (though  not  the  only 
possible  one)  of  detecting  Isolated  aberrant  readings  and  of  measuring  sev¬ 
eral  sorts  of  systematic  departures  from  the  Ideal  conditions.  That  ie  what 
this  paper  is  about— obtaining  information  concerning  conformity  with  the 
Ideal  conditions,  which  Is  a  necessary  step  before  criticizing  and  possibly 
improving  the  original  analysis. 

IH.  RESIDUALS  AND  FITTED  VALUES.  Corresponding  to  any  otosnetke, 
the  "fitted  value"  is  the  least-squares  eatimate  of  the  mean  value  of  the 
hypothetical'  chance  distribution  from  v^ch  the  observation  was  drawn, 
according  to  the  Ideal  conditions.  The  “residual"  Is  the  difference  between 
the  observation  and  the  fitted  value. 
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Our  example  of  the  penetration  data  has  the  peculiarity  that  there 
are  two  observations  in  every  cell  of  the  Latin  square.  -One  might  examine 
the  residuals  corresponding  to  these  200  individual  readings.  However, 
for  the  pufpose  of  comparing  rows  or  columns  or  letters.  It  Is  the  100  cell 
sums  y,.  that  are  relevant,  and  which  we  should  hope  would,  satisfy  the 
ideal  conditions  fairly  closely.  So  we  now  consider  these  as  the  effective 
observations  and  form  the  corresponding  100  fitted  values  (fy)  end  residual* 
(zjj).  Each  fitted  value  consists  of  the  sum  of  the  relevant  row  mean,  col¬ 
umn  mean  and  letter  mean,  minus  twice  the  overall  mean.  Per  example, 
corresponding  to  yjj(*  136.875)  we  have 

yr  »  123.69,  yml  -  128.73. .y^  -  129.36,  ?  -125.30, 
and  hence  the  fitted  value  is 

Yn -  yr  +  y*i  +  -  2y  -  131.18 

and  the  residual  la 

.  ' 4 

ZU  “  yll  "  Y11  "  5*69, 

When  the  fitted  values  and  the  residuals  corresponding  to  the  one- 
hundred  cell  sums  yjj  have  been  caloulated,  the  scatter  diagram  shown  in  Figure 
1  can  be  plotted.  Each  point  corresponds  to  one  of  the  cells  of  the  Latin 
square,  and  has  the  fitted  value  as  abscissa  and  the  residual  as  ordinate. 

Provided  that  no  error  has  been  made  in  the  calculation;  the  scatter 
diagram  must  have  the  properties 

*11  "  ^IJ  Zii  YIJ  ”  D: 

that  is,  the  average  of  the  ordinates  must  be  zero,  and  the  coefficient  of 
linear  regression  of  the  residuals  on  the  fitted  values  must  also  be  zero. 

If  the  ideal  conditions  are  exactly  satisfied,  the  diagram  should  have  that 
further  properties,  that  the  residuals  appear  in  aggregate  to  bo  normally 
distributed,  and  that  they  show  no  dependence  of  any  son  cm  the  fitted 
values. 


In  the  present  case  ohe  peculiarity  Is  immediately  noticeable,  thet 
the  residuals  have  a  negatively  skew  distribution;  they  range  from  ■»  11  to 
-  19,  roughly.  Another  peculiarity  is  easily  perceived  when  one  looks  for 
it,  namely,  that  the  vertical  dispersion  of  the  points  is  greater  on  the  left 
side  of  the  diagram  than  on  the  right.  Thus  the  three  largest  positive 
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residuals  and  the  six  largest  negative  residuals  ere  all  associated  with 
fitted  values  that  are  smaller  than  y  t-  125.3).  These  features  of  the 
scatter  diagram  suggest,  respectively,  that  if  the  observations  are  thought 
of  as  having  a  chance  distribution,  then  the  distribution  must  be  negatively 
skew  rather  than  normal,  and  that  the  variance,  Instead  of  being  constant, 
is  Smaller  when  the  cell  mean  Is  greater. 

It  Is  not  the  c&se  here  that  any  one  residual  Is  so  much  larger  In 
magnitude  than  the  others  as  to  suggest  a  gross  error  or  blunder  In  the' 
corresponding  y.  That  is,  there  Is  no  clear  outlier,  and  we  are  not  tempted 
to  reject  any  observation  as  spurious. ' 

Another  effect  which  may  sometimes  be  seen  in  such  a  scatter  dia¬ 
gram,  but  is  not  seen  here,  is  a  quadratic  or  curvilinear  regression  of  the 
resLduais  on  the  fitted  values.-  We  remarked  above  that  there  is  necessarily 
no  linear  regression  of  residuals  on  fitted  values,  but  a  nonlinear  regression 
is  not  precluded.  Such  a  regression  can  arise  If  the  effects  of  rows,  coluiqns, 
and  letters  are  not  additive,  in  the  way  stated  in  the  Ideal  conditions.  In  fact 
here  only  the  rows  have  a  substantial  affect.  Columns  seem  to  have  a  rather 
slight  affect,  and  letters  no  effect  at  all.  There  is  therefore  not  much  scope 
for  nonaddltivity,  and  It  Is  not  surprising  that  no  curvilinear  regression  Is 
nottcable.  . 

To  supplement  the  Visual  inspection  of  the  scatter  diagram,  one  may 
calculate  various  measures  of  departure  from  the  ldeel  conditions,  and  make 
significance  tests  and  other  assessment .  Relevant  formulas  are  given  In 
Cs]  .  For  example,  in  the  present  case,  one  may  estimate  a  measure  of 
skewness  (  in  Karl  Pearson's  notation,  •/.  tnR.  A.  Fisher's  )  of  the 
.piesumed  common  distribution  of  deviations  of  the  y’s  from  the  Unaer  cell 
means.  Thei  estimate  comes  out  at  -0.96,  with  standard  error  under  the 
full  Ideal  conditions  roughly  0.39. 

To  sum  up,  inspection  of  the  residuals  end  thelT  relation  with  the 
fitted  values  suggests  that  the  deviations  of  the  y's  from  the  cell  means  have 
a  skew  distribution  with  nonconstant  variance.  The  physical  cause  for  this 
is  no  doubt  that  occasionally,  perhaps  because  of  stones,  the  ground  it  so 
hard  that  the  penetration  Is  considerably  short  of  the  mean.  On  the  other 
hand,  there  is  no  reason  why  penetrations  much  in  excess  of  the  mean  should 
be  observed,  and  in  fact  because  the  rocket  motor  Is  broader  than  the  rod 
below  it  there  is  an  effective  upper  limit  to  the  depth  of  earth  penetration 
achievable  -  though  there  is  no  definite  evidence  in  these  observations  df  any 
piling  up  of  frequency  at  such  a  limit. 
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Do  these  phenomena  matter,  and  are  Tables  2  and  3  misleading? 
The  correct  answer  is  probably  no,  because  the  violation  of  the  tdealccn- 
dltlona  Is  not  extreme.  But  If  computations  are  done  automatically  and  with 
little  personal  effort ,  It  Is  worth  while  to  try  transforming  the  observations 
In  some  way  to  Improve  their  conformity  with  the  Ideal  conditions. .  Raising 
the  readings  to  a  power  greater  than.  1  is  suggested.  .This  would  be  parti¬ 
cularly  natural  If  there  were  theoretical  or  experimental  evidence  that  the 
propelling  charge  required  to  achieve  a  given  average  penetration  was  pro¬ 
portional  to  some  power  of  the  penetration;  It  would  then  be  natural  to- use 
that  power  here.  There  are  too  few  observation  to  fix  «n  appropriate  power 
closely,  from  examination  of  the  observations  only  -  a  rather  high  power,  •. 
sixth  or  seventh,  is  suggested. 

Let  us  consider,  conservatively,  raising  the  observations  to  the 
fourth  power..  That  Is,  all  200  original  rending*  are  raised  to  the  fourth 
power  and  divided  (for  convenience)  by  10&:  and  then  the  previous  analysis 
is  repeated.  We  find  that  the  skewness  measures  calculated  from  the  resi¬ 
duals  Is  now  about  halved  (-0.54),  and  moat  of  the  'egression  effect  of 
variance  on  cell  means  has  disappeared.  In  place  of  Table  3  we  have  Table 
4.  The  variance  ratios  are  nut  vastly  different  from  those  of  Table  3.  The 
block  and  plot  effects  have  emerged  a  little  more  distinctly,  end  there  is 
still  no  Indication  o!  real  difference  between  the  propellent  lots.  Table  4 
may  be  Judged  to  be  a  fairer  summary  of  the  effects  present  then  Table  3, 
but  evidently  our  conclusions  will  not  be  much  different  whichever  we  ex¬ 
amine.  It  would  be  desirable  to  Investigate  penetration  records  from  a 
number,  of  other  trials  before  venturing  on  a  general  recommendation  for  the 
statistical  analysis  of  such  data. 

1  am  Indebted  to  Mr.  JohnJ.  Simon  and  Mr.  Carl  E.  Jukkola  for 
carrying  out  the  compute tione. 
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Capital  Letter*  -  Propelling  Charge  lots  (A-J)r. 

Plot*  -  Plot*  of  ground  on  vfalch  test  vs*  conducted  (Each  plot  about 
12'  *  15’). 

Block*  -  Firing*  conducted  during  the  um  period  of  tlmu 
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NOTE:  Tvo  observations  per  cell  representing  two  propelling  charges  fro* 
n  lot.  Hole*  are  36”  apart. 
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Lot  A  S  C  D  X  J  0  H  I  J 
Penetration.  62.3  62.2  62.9  6l.8  Cl.O  64.7  6l.7  62.8  63.2  62.9 
Estinated  Standard  error  of  each  assn  -  1.26. 


TABtS  3 

Analysis  of  variance  of  penetrations 


Degrees  of 
Preedon 

Bun  of 
squares 

Mean 

squares 

Analysis  of  sumo  of  cell  pairs 

» 

• 

Between  blocks 

9 

2225 

247.3 

Between  plots 

9 

1376 

152.8 

Between  propellant  lots 

9 

2B3 

31.4 

Residual 

.72 

4579 

63.6 

Analysis  of  differences  of  cell  pairs 

* 

.  Total 

100 

3076 

30.8 

TABS  4 


Analysis  of  variance  of  penetrations  after  fcurth-pover  transformation  * 


Degrees  of 

Sum  of 

.  Mean 

■  •  -  ■ ,  : 

freedom  - 

squares 

squares 

Analysis  df  sums  of  cell  pelrs 

Between  "blocks 

9 

2164 

240.4 

Between  plots 

9 

1322 

146.8 

Between  propellent  lots. 

9 

256 

20.4 

Residual 

72  . 

3787 

52-6 

Analysis  of  differences  of  cell  pairs 

Total 

•  100  • 

2950 

29«5 
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of  some  trajectory  mz*z 

INSTRUMENTATION  SYSTEMS 


IS 


Oliver  Lee  Kingsley 

Range  Instrumentation  Development  Division, 
White  Sands  Missile  Range 


I.  INTRODUCTION.  The  purpose  of  the  analysis  was  to  isolate  the  random 
and  bias  errors  inherent  in  the  trajectory  instrumentation  systems  currently 
in  use  at  WSMR.  The  preliminary  analysis  presented  here  is  but  the  first  to 
be  made  on  a  series  of  missile  flights. 


Data  from  the  first  flight  is  still  undergoing  further  study. 

The  second  flight  of  the  current  series  was  made  September  1960  and 
the  analysis  will  commence  as  the  data  becomes  available. 


The  user  is  constantly  requiring  more  and  better  data.  These  require¬ 
ments  must  be  met  as  the  missile  systems  become  more  refined.  It  is  expected 
that  these  tests  can  lead  to  improved  instrumentation  systems  at,WSMRand  ' 
other  ranges. 

The  instrumentation  systems  used  for  the  initial  launching  were: 
Ballistic  Camera,  Askania  Cine-Tteodallte,  DOVAP  (Doppler  Velocity  and 
Position),  and  FPS-16  Radars.  Later  it  is  planned  to  include  the  Integrated 
Trajectory  System  (ITS)  in  the  series  of  tests.  The  ITS  is  a  system  capable 
of  simultaneous  multiple  object  tracking  by  combining  range  and  angle  in¬ 
formation.  The  range  and  angle  measurement  involves  the  use  of  electro¬ 
magnetic  phase -measuring  systems. 

Briefly,  the  analysis  will  cover  the  methods  used  to  estimate  the 
precision  of  each  Instrumentation  system  and  the  bias  of  each  instrumentation 
system. 

II .  PRECISION  AND  BIAS  ERROR  ESTIMATES.. 

A.  Precision  Estimates  by  Multi -Instrument  Method. 

The  first  attempts  at  precision  estimates  were  confined  to  the  variate 
difference  technique.  Later  the  multi-instrument  method  was  applied  as  data 
from  other  instrumentation  systems  became  available.  The  latter  method  has 
become  known  as  the  "Simon -Grubbs  Technique"  at  White  Sands  Missile 
Range  because  of  the  articles  by  General  L.  E.  Simon  and  Dr.  F.  E.  Grubbs 
illustrating  this  technique.  This  may  be  illustrated  briefly  by  the  assumed 
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mathematical  models  of  simultaneous  1-th  paired  measurements  y^  and  yi2 
from  instruments  #1  and  #2  respectively:  l"  " 


tu  vu  *  xu  *  bn  +  *u 


(2)  V(2  “*12  +  b12  +  *12 

where:  (a)  Xj.  *  and  represents  the  variability  of  the  1-th  quantity  or 
characteristic  Ming  measured. 


(b)  b..  and  b.,  represent  the  measurement  bias  of  Instruments  tl 
and  #2  respectively  while  measuring  the  i-th  quantity. 

(c)  etj  and  #l2  represent  the  random  error  of  the  i-th  measurement 
with  respect  to  Instruments  1  and  2 . 


Now  if  we  have  "n*  of  these  paired  measurements  we  may  form  *n" 
differences  of  the  corresponding  pairs.  Typically: 


(3)  (•„  *  «12>  ♦  *  bu) 

If  the  bias  is  constant  for  any  ”n"  paired  measurements,  we  can 
estimate  the  variance  of  y^  and  yJ2  as: 

(4)  *Y1  -  +  *2l 

<5)  'w  •  *x  +  'iz 

The  estimate  for  the  set  of  difference  is: 

(6)  ,d"*el*,e2* 


It  is  now  possible  to  estimate  the  instrumentation  variability  from  the 
equations  (4),  (S)  and  (€).  This  method  has  a  few  shortcomings.  Many  times 
one  achieves  negative  variance  estimates  which  require  some  interpretation 
Dr.  w.  A.  Thompson  has  worked  on  the  problem  of  negative  components  of 
variance  and  I  note  that  he  Is  scheduled  for  a  paper  on  the  subject  at  this 
neetlng . 


The  two  instrument  problem,  in  general,  cannot  be  applied  to  trajectoiy 
data  because  the  characteristic  measured  is  extremely  large  and  variable 
Compared  with  the  instrumentation  system  errors.  The  variance  of  the  esn 
mdhe^ioSt^mtnTalion  Variance  contains  the  estimated  variance  of  the 


r h*rjc1©r»ctic  i.€  (51,  for  <*st.  (CT^,)  we  have 

y.  e  1 


Best 
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2  , 


(7) 


2 

n-1 


) 


If  one  knows  the  ratio  of  precision  for  the  two  instrument  cose,  then, 
the  two  instrument  case  could  be  solved  for  precision  estimates.  This  is 
uaually  not  the  case. 

Estimates  of  the  error  variance  components  were  obtained  for  the 
four  instrumentation  systems:  Ballistic  Camera,  DOVAP,  Askania  and  Radars. 
The  square  root  of  the  variance  estimates  are  presented  in  table  2,  with  the 
exception  of  the  DOVAP  x-component.  The  variance  estimate  for  the  x- 
component  was  smell  and  negative:  thus,  the  variance  component  was 
equated  to  zero. 


Table  2:  Standard  Deviation  Estimates  by  the  Multi  - 
Instrumentation  Method. 


Coordinate 

Component 

_Ssttmeted 

Standard  Deviation  Estimate  ♦ 

_ Instrumentation  System 

.Belli  sue 

DOVAP 

Askenla 

Rader 

x  (feet) 

2 

0 

11 

15 

y  (feet) 

6 

4 

11 

21 

_ Ifi _ 

_ 8 _ 

8 

12 

on  28  consecutive  trelectorv  date  points.  1 

Other  estimates  can  be  obtained  from  the  analysis  of  variance  tables 
where  the  Ballistic  Camera  is  considered  as  a  standard  for  comparison. 

g£g£i&laa  StUmates  by  the  Variate  Difference  Method. 

The  variate  difference  method  was  applied  to  data  from  theDOVAP 
and  FP8-16  radar  systems  for  the  trajectory  segment  covered  by  Ballistic 
Camera.  Data  sampling  rates  for  the  DOVAP  and  the  FPS-16  systems  were 
much  higher  than  for  the  Askania  and  Ballistic  Camera  systems  which  were 
dependent  on  a  flashing  light  at  one  per  second.  Thus,  the  DOVAP  and 
Radar  systems  were  more  suited  to  this  technique. 

Table  3  presents  the  standard  deviation  estimates  for  the  DOVAP 
"  ’em  The  estimates  are  based  or.  second  difference. 
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Table  3:  Standard  Deviation  Estimates  by  Variate  Difference  Method. 


Nominal  time  along  trajectory 
segment  baaed  ontnlsslle 


DOVAP 

Standard  Deviation  Estimate* 
Coordinate  for  Component* 


liftoff. 

jctft.y 

Y  (It  .1 

*  (ft.) 

40*50  seconds 

0.17 

0.31 

0.24 

50-55  seconds 

0.17 

0.36 

0.26 

60-6.5  seconds 

0.23 

0.40 

0.31 

♦Each  is  based  upon  50  consec 

uttve  tralector 

y  data 

point.. 

|W 

'/."p’j- 

j-JS  kV^ 

P--  i 


These  estimates  filter  out  linear  noise  from  the  data  end  hence  ar» 
much  smaller  then  the  Simon-Grubbs  estimates  of  the  previous  section 
which  do  not  filter  the  linear  noise.- 


The  variate  difference  technique  v/a s  also  applied  to  trajectory 
data  available  from  the  FPS-16  radars.'  Eachradar  Was  analyzed  separately 
and  in  its  natural  coorc'  .nate  system:  range,  azimuth,  and  elevation.  The 
radars,  for  the  most  part,  exhibit  estimates  close  to  the  design  Intent: 
range  +  4  yards,  azimuth  +  0,1  mils,  and  elevation  4  0.1  mils  (these  are 
rms.  values)  when  evaluated  by  this  method.  Data  from  the  three  FPS-16 
radars  that  tracked  most  of  the  trajectory  are  shown  In  fable  4.  These 
data  cover  essentially  the  same  trajectory  segment  as  data  in  the  preced¬ 
ing  section. 

Table  4;  Precision  Estimates  by  Variate  Difference  Method. 


M 

p 
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Table  5:  Precision  Estimates  by  Variate  Difference  Method. 


Tracking 

FPS-16 

Radar 

Time 

Segment 

In  Seconds 

Standard  Deviation  Estimate 

Ranee  Azimuth  Elevation 

(Yds) 

(Mils) 

(Mils! 

112 

.8-100  .. 

2.68 

0. 31  - 

fL-21 

114 

10-100 

2.78 

0.15 

.  0.15 

122 

10-100 

1.27 

-0.3S 

0.20 

,G.  Svatem  Estimates  of  the  Bias  Error. 

Two  disjoint  trajectory  segment*  were  expected  for  the  Ballistic 
camera  coverage.  Each  of  these  segments  were  to  be  divided  into  a  first 
portion  and  a  last  portion.  Actual  missile  trajectory  segment  was  covered 
in  one  continuous  segment  from  approximately  39  seconds  of  flight  to  66 
seconds  of  flight  from  missile  lift-off.  Thus,  four  sets  of  seven  trajectory 
data  points  each  were  formed. 

At  simultaneous  times,  the  reduced  trajectory  data  from  Askanias, 
DOVAP  and  radars  were  each  differenced  with  respect  to  the  Ballistic  camera 
date.  The  set  of  error  difference  data  were  used  In  the  analysis  of  variance. 
The  Ballistic  Camera  data  were  considered  as  the  reference  standard. 

The  analysis  of  variance  of  the  DOVAP  difference  data  indicated  a 
significant  shift  In  the  bias  for  the  X  and  Z  component  segment  means.  The 
Y  component  of  DOVAP  data  Indicated  no  shift  In  the  means  for  segments. 
However,  a  significant  bias  is  indicated  in  each  of  the  overall  meant  coor¬ 
dinates  when  compared  with  the  expectation  of  zero.  Table  6  shows  the 
estimated  means  for  each  trajectory  segment  and  coordinate.  To  compare 
directly  with  the  Ballistic  Camera  data,  these  data  need  a  nominal  adjust¬ 
ments  in  each  coordinate;  the  largest  adjustment  is  approximately  8  feet  for 
the  X  component.  These  adjustments  do  not  change  any  of  the  above  conclu¬ 
sions. 

Table  6:  DOVAP  Mean  Bias  Error  Estimates. 


Component 

Coordinate 

-  Trajectory  Seqme nt 

Over-ell 
Mean  Bias 

1 

2 

3 

4 

X  (ft)* 

54 

49 

44 

42 

46 

Y  (ft) 

-17 

424 

-20 

-16 

-20 

z  (ft) . 

-64 

-81 

-80 

-92 

-B0 

I.  *X  needs  a  nominal  8  foot  adiustment  1 

as 
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The  analysis  of  variance  for  the  Askanla  errors  show*  a  significant 
shift  in  the  bias  between  the  trajectory  segments  for  each  coordinate  studied. 

In  addition,  there  is  a  significant  bias  in  the  overall  mean  for  each  of  the  • 

coordinates.  These  biases  are  undergoing  further  study  at  the  present  time.  * 

The  estimated  mean  error  for  each  segment  is  shown  in  Table  7. 

Table  7:  Askant,  Mean  Bias  Error  Estimates. 


Component 

Coordinate 

Over-all 
Mean  ...... 

l 

,  2 

am 

4 

„X  (ft) 

13 

10 

8 

_ 6.4 _ 

hnm 

3 

wsm 

-2 

.  r 

-20 

HHEXJHIIR 

L^.  z  M. 

-22 

wmm 

-44 

The  FPS-16  radars  exhibit- a  significant  shifting  mean  along  the  tra¬ 
jectory  in  the  Z  coordinate.  However,  the  overall  mean  error  for  the  Z 
coordinate  Is  not  significantly  different  from  zero.  For  the  "X"  and 
coordinate,  the  overall  means  exhibit  a  significant  bias.  This  is  not  being 
investigated  further  because  the  tracking  was  not  a  point  source  such  as  a 
beacon.  A  Beacon  track  was  intended  for  the  shoot  but  was  not  attained. 
Table  8  below  exhibits  the  mean  data. 

Table  8:  Radar  Mean  Bias  Error  Estimates. 


Component 

Coordinate 

Tra 

ectorv  S.eament 

■*naii 

1 

_ 2 _ I 

3 

4  ..  .. 

Mean 

_ x  (ft) 

KM 

-17 

-3 

-11.5 

_ Ylft) _ 

in 

HS 

8 

25 

16.8 

nna 

U  2 

LlLlJ 

20 

. -L..., 

3.6 

I 
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TaMre  9:  DCWAF  Analyst*  oTVatlance  TMtflws Her 'Traftetttpty  ’Svynwit*. 


Sources  of  Variance 

d.f. 

s.s. 

m.a. 

F 

S,  -Traiectorv  Seament 

3 

'389 

129 

7.58 

<  -  Error 

24 

406 

17 

Totals 

27 

795 

{  -Trajectory  Segment 

3 

284 

_  95 

3.16 

f. -  Error  . . 

24 

.  713.  , 

.  30 

Total 9 

27 

,  997 

Z.  -Tra  lector/.  Segment... 

3 

2691 

897 

28.9 

7,  -  Error 

24 

748. 

: . . 3i . , 

Totals 

27 

3439 

Table  10:  Askanta  Analysis  of  Varli 

race  Tables. f< 

yr  Trajectory  Segments. 

Sources  of  Variation 

d.f. 

s.s. 

m.«. 

F 

If  -Trajectory  Segment 

3 

1414 

471.3 

7.01 

)C  -  Error 

24 

1614 

67.2 

'  1  '  ■'  J 

Totals 

27 

3028 

i  -  Trajectory  Segment 

3  .  _ 

,20.48 . 

.  682.5 

7.21 

l  -  Error 

24 

2275 

. . .  94.7 

Totals 

27 

_  4323 

Z  -Trajectory  Segment 

3 

3035 

1011.7 

10.10 

2  -  Error 

24 

2406 

100.2 

Totals 

27 

5441 

■ -  -  -  —  -  -  — - -  - _ l . . '  1-  -  »  '  ' 

Table  11:  Radars  Analysis  of  Variance  Tables  for 

Trajectory  Segments. 

Sources  of  Variation 

d.f. 

s.s. 

m.s. 

F 

f  -Trajectory  Segment 

3 

880 

293 

.  1.71 

(  -Error 

24 

4093 

171 

Totals 

27 

4973 

l  -Tralectorv  Segment 

3 

1149 

383 

{  -  Error 

24 

11980 

499 

Totals 

27 

13129 

!  -Trajectory  Seqment 

3 

3212 

1071 

8.74 

2  -  Error 

24 

2957 

123 

Totals 

27 

6159 
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01.  CINE -THEODOLITE  HIM  SHADING  PRECIST QK.  An  example  of  sub¬ 
system  study  is  given  by  this  tracking  correction  digression.  Each  cine- 
theodolite  record  was  read  by  three  different  reading  personnel.  The  set 
of  readings  with  the  lowest  reader  variance  was  used  in  the  data  reduction 
process.  Table  12  gives  the  precision  estimates  of  the  tracking  correction 
readings. 

Table  12:  Precision  Estimates  for  Eight  Cinetheodolite  Records.  . 


Film 

Coordinate 

System 

Standard  Deviations  tof  Tracking  Corrections 
Cinetheodolite 

1 

.  2 . L  3, 

4 

5 

6 

7 

e 

X,  (arc  sec)  . 

3.0 

.  M  l  0,4 

0.4 

2,g 

3tl 

3,8 

2,8 

,  Y  (arc  one).  _ 

lltiL 

JLA. 

2J. 

2J 

The  estimates  were  derived  by  the  three -instrument  method  (Simon- 
Grubbs).  The  sample  sixes  ranged  from  a  low  of  34  X-Y  pairs  to  the  maximum 
of  99  X-Y  pairs.  There  is  a  trend  indicated  for  low  estimates  in  X  to  be 
paired  with  low  estimates  in  Y.  This  la  to  be  expected  because  a  good  film 
record  could  be  read  well  In  either  coordinate. 

IV.  SUMMARY  AND  CONCLUSIONS. 

A.  The  DOVAP  end  Ballistic  Camera  are  among  the  most  precise  systems  in 
jse  at  WSMR.  The  DOVAP  has  the  shortcoming  that,  in  general,  the  tra¬ 
jectory  data  are  biased  from  the  true  trajectory. 

3.  The  shift  in  bias  along  the  trajectory  was  significant  in  all  coordinates 
for  Askanla  data;  significant  in  the  X  and  Z  coordinate  for  DOVAP  data;  and 
significant  In  the  Z  coordinate  for  Radars. . 

C.  There  was  a  significant  overall  bias  In  all  coordinates  except  for  the 
Raders  in  the  Z  coordinate. 

D.  The  significant  biases  in  the  DOVAP  and  Askanla  Instrumentation 
.systems  should  be  studied  and  mathematical  or  physical  methods  developed 
to  remove  them. 
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MODEL.  TO  AEROSOL  CHAMBER!  TRISE  DA’EL 

TheodoeeW.  Bom«r 
Boo*,  Allen  Applied  Research,  Inc, 

In  an  aerosol  cloud  release,,  a  bacterium  may  be  Ineffective  because 
of  death  before  some  critical  time.  This  bacterial  decay,  as  it  ts  called, 
hat  been  studied  by  meant  of  chambef.  trials  by  releasing  a  cloud  into  the 
chamber  and  then  sampling  the  chamber  at  periodic  intervale.  One  of  the 
unique  sett  of  data  in  this  regard,  from  the  standpoint  of  precision  and 
amount  of  replication,  la  that  auppllsd  by  -Dr.  T.  L.  Snyder  and  Hugh  Lea, 
of  Port  Detrick. 

In  theta  trials,  an  aerosol  cloud  of  particles  containing  baateria 
and  tracer  material  was  generated  Inside  a  chamber..  Ten  pairs  of  samples 
wets  withdrawn  from  the  chamber  at  half-minute  Intervals  starting  at  the 
first  half  minute.  A  bacteria  count  waa  obtained  on  one  sample  of  each 
pair  and  a  tracer  measurement  on  the  other.  Estimates  of  Initial  tracer  end 
bacteria  counts  were  alfco  obtained  using  a  knowledge  of  the  composition  of 
the  spray  material,  spray  rate  and  duration.  Traoer  materiol  waa  included 
In  the  cloud  release  because  the  particles  on  which  the  bacteria  were  loca¬ 
ted  were  continually  falling  to  the  bottom  of  the  chamber  and  the  tracer  dati 
were  used  to  correct  for  this  fallout  loss.  The  corrected  date,  which  ere 
described  as  biological  recovery  percentages,  were  computed  for  each 
half-minute  Interval  aa, 

r  -  loo  x  3s. 

*,  •  T 

where 

Ba  is  the  sample  bacteria  count; 

Bj  is' the  initial  bacteria  count; 

Ts  is  the  sample  tracer  measurement; 

Tj  is  the  initial  tracer  measurement. 

Chamber  trials  were  ran  at  relative  humtdlttes  of  12,  36,  62,  and  B7%  with 
a  medium  1,  and  at  12,  19,  36,  49,  65,  and  86%  with  a  medium  2.  About 
twelve  trials  ware  conducted  at  each  relative  humidity,  medium  combination'. 

One  of  the  unique  features  of  the  Snyder-Lee  data  is  shown  in  Slide 
1,  which  shows  plots  of  viable  recovery  percentages  versus  time  on  log-log 
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paper,  'tfttih -dl  these  tuam&ts  ere  ixwrrtKlittadl  wrt ie»  the  dsi*  of  metfitasni  l;  the 
lower  corresponding  to  a  chamber  relative  humidity  of  36%  and  the  upper  to 
a  chamber  relative  humidity  of  1291.  The  plotted  points  are  Overage  biologi¬ 
cal  recovery  percentages  taken  over  all  similar  trials.  The  upper  curve  la 
concave  downward  and  la  typical  of  the  typa  of  curve  that  is  observed  for 
all. of  the  relative  humidity,  medium  combinations  except  the  one  .associated 
with  the  lcwer  curve,  which  la  concave  upward. 

Several  models  have  been  proposed  by  Dr.  Snyder  and  others  for 
these  biological  recovery  curves.  The  Welbull  model  was  found  to  give  an 
excellent  fit  in  all  cases  except  the  data  for  medium  1,  relative  humidity 
36%.  The  Weibull  model  will  not  give  concave  upward  curves  in  log-log 

space. 

A  model  which  did  give  a  good  fit  to  the  data  for  all  of  the  medium, 
relative  humidity  combinations  was  the  exponential  hazard  model.  This 
model  is  defined  as, 

R  *  exp  [a  +  b  exp  (-ctQ 

where  R  la  biological  recovery  percentage  at  time  t,  and  a,  b,  and  c 
are  constants.  Hazard  rate  H(t)  is  defined  as  the  chance  that  a  bacterium 
will  die  In  the  lntorval  dt  given  that  It  has  lived  to  time  t.  The  hazard 
rate  for  the  exponential  hazard  model  la 

H(t)  »  -  (1  'RJ  ■  bo  exp  (-ob) 

which  will  plot  as  a  straight  line  on  semi-log  paper.  The  model  was.  In 
fact,  suggested  by  observing  the  computed  point -by -point  hazard  rate 
plots  on  semi-log  paper.  This  hazard  rate  1«  to  be  contrasted  to  a  constant 
hazard  rate  for  exponential  decay  for  biological  recovery  percentages  and  ■  ' 
hazard  rate  of 

H(t)  - 

for  the  Wj^Jbull  model. 

When  the  exponential  hazard  model  is  plotted  on  log-log  paper,  the 
biological  recovery  percentage  curve  Is  concave  downward  for  l/c  and 
concave  upward  for  t>l/c.  The  initial  recovery  percentage  at  t  ■  0  is 
exp  (a  4  b)  and  the  recovery  percentage  at  t  ■  <**  is  exp  (a) .  This  model 
was  fitted  to  the  Snyder-Lee  data  by  computing  the  regression  of  Y  ■  Inll 
on  X*  exp  (-ct),  choosing  c  so  as  to  minimize  the  sum  of  squares  of  the 
deviations  from  regression. 
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Tire  rtndto  at  a&Hiss*  Dh*  exjpnowattet  fcasawi  model  to  averages 
over  similar  trials  of  the  viable  recovery  percentage*  are  shown  In  Slide 
2.  The  model  was  not  fitted  to  the  date  of  medium  1  at  the  relative  humi¬ 
dities  of  62  and  87%,  because  there  was  no  evidence  of  bacterial  decay1 
within  the  time  span  covered  by  the  data  apart  from  an  initial  decay  of  5 
to  10%.  In  the  right-hand  column  of  the  slide  era  the  percentages  of  the  Y 
sum  of  squares  accounted  for  by  linear  regression  of  Y  on  X.  These  per¬ 
centages,  with  two  exceptions,  are  above  99%.  The  exceptions  occur  at 
the  high  relative  humidities  for  medium  2  and  are  due  to  the  fact  that  the 
slope  of  the  regression  line  at  these  higher  relative  humidities  is  so  gradual 
that  the  regression  sum  of  squares  becomes  small  relative  to  the  noise 
which  is  present.  This  fact  is  pointed  up  perhaps  batter  by  Slide  3,  which 
•hows  the  plots  of  Y  ■  In  R  versus  X  -  exp  (-ct)  .for  the  different  relative 
humidities  and  mediums.  Looking' at  the  upper  line  for  medium  2,  relative 
humidity  86%,  the  noise  did  not  appear  greeter  here  than  at  other  relative  ' 
humidities,  but  the  shallow  slope  of  the  line  materially  reduced  the  sum  of 
squares  accounted  for  by  regression. 

.  Slide  4  shews  other  cases. ,  Hare  again,  the  exponential  hazard 
modol  dld  not  appear  to  contradict  the  date. 

The  variation  of  para  meters  of  the  model  with  relative  humidity  ia 
.  shown  In  Slide  S.  The  horizontal  axle  it  associated  with  relative  humidity 
and  the  vortical  axis  with  parameter  value.  In  the  case  of  the  a  and  b 
parameters,  a  discontinuity  appears  to  occur  in  the  neighborhood  of  a  rela¬ 
tive  humidity  of  45%.  The  a  and  b  curves  appear  to  be  almost  mirror 
images  of  each  other.  This  is  probably  because  the  sum  of  a  and  b  is 
associated  with  initial  recovery  and  hence  ts  a  more  fundamental  parameter. 
For  a  particular  relative  humidity,  the  sum  of  a  and  b  can  be  read  off 
the  graph  and  used  as  an  entry  in  the  lower  right  table  in  the  graph  to  find 
the  initial  recovery  percentage.  Thus,  the  sum  fer  a  relative  humidity  of 
21%  Is  3.6,  which  la  associated  with  an  initial  biological  recovery  of  37%. 

The  variation,  of  the  initial  biological  recovery  percentage  with  rela¬ 
tive  humidity  is  not  too  surprising.  However,  the  nonzero  recovery  percent¬ 
ages  et  time  equal  infinity  are  somewhat  more  suspect,  Thus,  when  the 
relative  humidity  is  58%,  the  predicted  value  for  the  a  parameter  is  2,  and 
using  the  lower  left  table  on  the  graph,  recovery  percentage  at  t  ■  «*°  is' 
estimated  at  7.39%.  This  nonzero  final  recovery  percentage  is  a  matter 
which  is  subject  to  experimental  verification,  although  such  verification  la 
not  possible  with  present  sets  of  data,  because  of  the  limited  time  span  and 
relative  humidity  levels  covered  by  the  data. 
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.although  It  Is  mat  too  tflaar  itn  iBIifle  V,  c  declines  from  «  value  of 
about  D -  31  at  W%  relative  humidity  to  about  0.20  at  65%  relative  humidity. 
The  odd  value  of  c  “  0 . 80  for  the  relative  humidity  level  of  B6%  wa*  prob¬ 
ably  a  poor  estimate  due  to  the  noise  occurring  at  this  humidity  level.  In 
this  case,  the  sum  of  squares  would  not  be  substantially  changed  regardless 
of  the  c  value  used. 

Using  the  graphs  which  relate  parameter  values  to  relative  humidity 
level  and  the  exponential  hazard  morel,  It  was  possible  to  construct  graphs 
relating  the  three  quantities j  relative  humidity  level,  psr  cent  recovery,  and 
time,  and  from  these  graphs  make  predictions  that  could  be  uesd  for  further 
testing  of  the  model. 

Slide  6  Indicates  the  results. of  fitting  the  exponential  hazard  model  ' 
to  individual  trials.  The  trials  are  ihofie  for  medium  2,  relative  humidity 
36%.  For  these  particular  trlcla,  the-expowtntial  hazard  model  explained 
substantial  portions  of  the  variability.  The  manner  in  which  estimated  initial 
recovery  percentages  varied  from  trial  to  ttlal  is  shown  in  the  column  labeled 
t  *  0.  Apart  from  trial  1133,  which  Indicates- an  initial  biological  recovery 
percentage  of  61%,  the  percentages  vary  from  a  low  of  14%  to  a  high  of  30%. 
Part  of  this  variation  may  be  due  to  Imprecise  control  of  the  relative  humidity 
of  the  chamber. 

As  regaida  the  exponential  hazard  model,  there  are  a  number  of  areas 
that  require  further  investigation;  namely: 

1.  Validity  of  the  model,  - 

2.  Theoretical  implications  of  the  modal, 

3.  Application  of  the  model  as  a  research  tool. 


Graphi  of  Average  Recovery  Percentage*  on 
Log-Log  Paper  for  Orguniarii  1,  Medium  l 
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Heaults  of  Fitting  the  Exponential  Hazard  Model 
to  Snyder-Lee.  Data  on  Average  Recovery 
Percentages  Over  Similar  I’riale 
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Relationship  of  Parameters  to  Relative  Humidity 
Level  for  Snyder-Lee  Data  for 
Organism  l.  Medium  2 
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Slide  6 

Results  of  Fitting  the  Exponential  Hazard  Model  to 
the  Trials  of  Medium  2,  Relative  Humidity  36% 


Estimate*  of  Recovery 
Percentages ' 

Percentage  of  Y  Sum 
of  Squares  Accounted  for 
by  Linear  Regression 

Initial 

.  Final 

Trial 

4  •  0 

t  ■  • 

of  Y  on  X 

T5 

16.69 

.  039 

69. 68  . 

76 

14.68' 

.033 

98.69 

ommisu'’  <se  ileinst  maoii:  v&zvzn:  iHgaaxngfc 

John  £.  Mslllgo 

U.  S.  Army  Chemical  Corps  Biological  Laboratories 
Fort  Detrick,  Frederick,  Maryland  v 


In  the  study  of  atmospheric  currents  and  the  behavior  of  particulates 
in  aerosol,  a  variety  of  finely  divided  materials  is  used  as  tracers.  An  im¬ 
portant  clasB  of  tracer  Is  the  fluorescent  particulates..  Among  the  fluorescing 
mineral  compounds,  the  sulfides  of  zinc  and  cadmium  are  particularly  useful. 
Capable  of  being  produced  lp  extremely  well  controlled  ranges  of  panicle 
size  and  detectable  in  minute  quantities,  these  sulfides  have  been  used 
widely  in  aerosol  studlss. 

A  typical  test  using  these  tracers  Involves  their  aerosolized  on  In  an 
atmospheric  system  of  interest  and  subsequent  sampling  at  a  time  and  place 
dictated  by  the  test  objectives.  Sampling  is  achieved  by  filtering  a  metered 
quantity  of  the  aerosol  through  a  membrane  filter  which  retains  virtually  all 
of  the  particles  contained  therein.  After  suitable  preparation,  the  filters  are 
assayed  visually  with  a  low  power  light  microscope  using  ultra  -violet  illumi¬ 
nation  to  Induce  fluorescence  in  the  particles.  This  process,  of  course,  en-. 
tails  counting  all  or  a  sample  of  the  particles  on  the  ftlter,  which  is  at  best 
a  tedious  job.  On  heavily  laden  filters,  the  errqrs  due  to  distribution  of  the 
particles  are  further  augmented  by  thpse  from  human  fatigue  and  confusion  of 
the  point  light  sources.'" On  filters  with  relatively  low  particle  densities,  the 
time  consumed  In  counting  an  adequate  number  of  particles,  say  300,  which 
Is  considered  a  reasonable  sample,  may  be  as  much  as  20  or  30  minutes. 

This  is  due  to  the  necessity  of  observing  a  large  number  of  microscopic  fields, 
each  . of  which  aontalna  only  a  few  particles.  In  a  large  scale  test,  with  pos¬ 
sibly  hundreds  of  filter  samples,  the  labor  Involved  becomes  excessive.  For 
this  reason,  much  thought  has  been  given  to  development  of  an  automatic 
fluorescent  particle  counter. 

No  machine  has  been  developed  to  count  these  particles,  as  such, 
but  the  General  Electric  Company  at  Hanford  Atomic  Works,  Richland,  Wash¬ 
ington  has  developed  a  device*  which,  by  detecting  scintillation  induced  In 
zinc  and  dadmlum  Bulfldes  by  an  alpha  emitting  source,  can  give  a  quanti¬ 
tative  estimate  of  the  man  of  the  material  present.  This  device  gives  data 
in  scaler  readings  of  the  number  of  nuclear  disintegrations  per  minute. 


*Dlscussed  by  M.  O.  Rankin  at  the  Meeting  of  the  American  Meteorological 
Society,  San  Diego,  California,  June  15-19,  1959. 
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The  problem  to  be  precepted  her#  concerns  the  attempt  to  calibrate 
thia  device  in  terms  of  particles  per  filter.  A  series  of  filters  was  pre¬ 
pared  with  graded  loadings  of  pertloles  to  cover  a  range  coneidered  of 
practical  value  (1.  e. ,  from  10  partlclea  per  filter  to  one  million  partlclea 
per  filter  In  ten-fold  lrora’usnts); 

Typical  examples  o!  the  machine's  reaponae  to  flltera  of  known 
particle  count  are  given  in  slide  1,  which  indicates  that  Its  threshold  is 
at  about- 100  particles  per  filter  end  that  no  apparent  upper  limit  of  useful 
response  was  reached,  It  la  background  radiation,  In  the  form  of  oosmlo  * 
rays,  etc.,  Which  essentially  determines  the  threshold  of  machine  sensi¬ 
tivity,  of  course. 

.  Slide  No.  2  presents  e  typical  set  of  data  in  which  visual  counts 
ere  plotted  against  machine  response  on  logarithmic  paper.  (Consider 
points  of  machine  1  only.) 

Our  approach  has  bean  to  fit  a  linear  regression  to  the  data  between 
the  limits  of  1000  to  one  million  particles  .per  filter.  Thia  equation  is  given 
In  the  next  slide.  (SlldeS.)  ■ 

The  question  to  be  asked  here  Is,  do  the  data  warrant  fitting  a 
curvilinear  regression  starting  at  the  100  particle*-  par  filter  level?  A 
corollary  question  Is  what  wbuld  be  the  statistical  validity  of  such  a 
curvilinear  regression,  considering  that  the  visual  counting  la  extremely 
precise  and  accurate  at  low  counting  levels  while  the  machine  response  la 
]ust  the  opposite. 
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SLIDE  3 


Y-  -0.7992  +1.03059  X  1 

standard  error  of  b  *  0.0090 
r2-  0.993 

Note:  b  -  slope 

r  •  correlutlon  coefficient 


SELECTS  •  QE-  AfKfTNG:  POINT  PATTERNS  ON  BOMB  SALVO  TARGET  COVERAGE 

Ralph  D.  Doner 


Systems  Ana lysis  Laboratory,  Requirements  and  Plans  Division. 
Research  and  Development  Operations,  Army  Rocket  and  Guided 
Missile  Agency,  Redstone  Arsenal,  Alabama 


1.0  PROBLEM  STATEMENT. 

1-1  PROBLEM  I.  To  obtain,  on  a  given  confidence  level,  the  most 
nearly  uniform  specified  fractional  coverage  of  a  large  homogenous  target 
.  area  by  varying  the  geometry  and  komb  allocation  of  multiple  aiming  points  fee 
s  salvo  of  bombs  having  a  fixed  radius  and  circular  error  probability. 

1.2  PROBLEM  H.  To  find  the  simplest  computational  techniques 
for  obtaining  acceptable  solutions  ler  Problem  I  cn  manual,  analogue  and 
digital  levels. 

2.0  ANALYSIS. 


2.1  CENTRAL  PATTERN  OPTIMIZATION.  Bombs  aimed  at  the  central 
points  of  the  pattern  w«ll  provide  most  of  the  coverage  for  the  central  portion 
of  the  large  homogenous  target  area.  Uniformity  in  this  coverage  calls  for 
uniformity  10  the  geometry  of  the  pattern  and  in  the  allocation  of  bombs  to 
indivldvial  points.  In  seeking  the  optimum  characteristics  of  the  central 
portion  of  the  pattern  the  analysis  will  progress  from  consideration  of  a  sin¬ 
gle  aiming  point  to  a  row  of  points  and  finally  to  two  dimensional  arrays. 

2.1.1  ONE  AIMING  POINT.  Traditionally,  the  coverage  for  this  case 
is  determined  in  terms  of  the  probabilities  of  hits  and  overlaps  and  the  rati » 
of  the  bomb's  lethal  area  to  the  area  cf  the  target.  Central  symmetry  about 
this  one  ah.  g  point  makes  coverage  a  function  only  of  the  radial  distance 
from  the  center  of  the  bomb  burst  to  the  aiming  point.  Double  integration  is 
not  inherently  unavoidable  In  empirically  determining  this  functional  rela¬ 
tionship.  A  discrete  set  of  concentric  circles  is  In  order,  each  to  serve  as 
an  isohap*  curve,  that  is,  as  a  contour  of  constant  probability.  Fig.  1 


*In  "Handbook  of  Probability  and  Statistics  with  Tables",  by  Burrington  and 
May,  Handbook  Publishers,  Inc.,  1958,  on  page  98,  the  expression  "equi- 
prcbability  curve"  is  used.  A  shorter  term  such  as  "isoaleatory" ,  or  better, 
"isohap",  is  desirable  if  this  concept  should  gain  wide  usage. 
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IHueaMoss  ■  tHffl  fltar  ten  borrtbf* .  &  tzftfe  rdf  'Gbub'BIbt  sflwifcttes  'in  unrwfi  to 
locate  the  centers  of  bomb  Impacts.  The  lethal  radius  of  the  bombs  l*  0.5®* 
and  the  lsohap  circles  ere  spaced  0.50*  apart.  The  restating  arc  coverage 
on  each  isobap  la  accumulated  on  the  centrally  pivoted  dividers,  shown  in 
the  figure  in  the  act  of  evaluating  the  coverage  bomb  number  6  hae  given 
the  2.0 <T  lsohap.  Totals  are  read  In  decimal  form  on  the  peripheral  scale. 
In  figure  1  are  given  the  coordinates  of  Impact  centers  end  the  epvemge  pro- 
file. 

2.1.2  A  ROW  CT  AIMING  POINTS.  Figure  2  shows  several  of  a  row 
of  aiming  points.  The  row  is  sufficiently  long  to  fully  account  for  the  hit 
probabilities  of  all  Intermediate,  points  on  this  line.  What  Is  needed  here 
Is  a  simple  index  of  the  resultant  hit  probability  on  a  specific  point  on  this 
continuous  line,  in  terms  of  its  distance  from  contributing  aiming  points.  • 
Bomb*!  have  a  bivariate  distribution  about  their  aiming  points.  In:  the  case 
of  circular  error  probability,  the  ordinates  of  a  bivariate  normal  surface  are 
approximately  0.4  times  the  ordinates  of  a  univerlato  normal  curve  at  equal 
distances  from  centers  of  symmetry  and  for  equal  standard  deviations  O'  • 
Both  therefore  hsve  their  inflection  points  one  unit  from  their  maximum 
points.  As  a  consequence,  ordinates  of  the  univariate  normal  curve  may  be 
used  as  indlcos  of  hit  probability.  Addition  of  these  ordinates,  as  shown  in 
Figure  2,  (where  curves  are  spaced  to  Intersect  at  their  Inflection  points), 
results  in  a  near  lsohap  that  fluctuatee  between  0.493  and  0.507.  Such 
near  uniformity  in  hit  probability  implies  near  uniformity  in  overlap  proba¬ 
bility.  The  occurence  of  overlap  on  a  bomb  drop  exercises  a  negative  feed¬ 
back  on  the  probability  of  overlap  for  subsequent  drops,  thereby  tending  to 
make  coverage  expeotnney  more  nearly  uniform  than  hit  probability. 

2-1.3  RECTANGUIAR  GRIP  OF  AIMING  POINTS...  In  Figure  3  part 
of  a  rectangular  grid  of  aiming  points  is  given.  Enough  points  are  shown  to 
account  for  the  total  hit  probability  Indices  of  a  central  point  X  and  of  two 
other  centers  of  symmetry  Y  and  Z.  Contributions  made  by  sets  of  aiming 
points  symmetrical  to  each  of  these  three  points  X,  Y  and  Z  are  tabulated 
in  the  figure  for  separations  S  from  1.8  to  2 >2.  A  separation  of  .9  appears 
to  be  near  optimum.  In  each  case  X  is  a  maximum,  Y  a  saddle  point,  and  Z 
a  minimum.  Repetition  of  these  values  at  similar  positions  throughout  the 
central  portion  of  the  pattern  provides  a  measure  of  the  degree  of  i.nlfurmlty 
in  expected  coverage  associated  with  thts  particular  geometry  witf  equal 
allocation  of  bombs  to  the  aiming  points. 

2.1.4  HEXAGONAL  GRID  OF  AIMING  POINTS.  Treatment  similar  to 
that  given  the  rectangular  grid  Is  indicated  in  Figured  for  the  hexagonal 
array.  The  separation  is  2.0  units,  and  again  X  is  a  maximum.  However, 


49 


Design  of  Experiments 

Y  and  Z  have  exchanged  Tlte  degree  of  ontfbwxittY  is  excellent,  lndi— 

cattttag,  that  die  rmctegonaU  'grti  la  the  'Opftfiamm  ipettem. 

2.2  narrrDV  PT.TUPHTirr  OPTIMIZATION.  The  peripheral  character¬ 
istics  of  the  target  obviously  determine  the  boundary  propertlee  of  the  pattern. 
The  circle  with  a  alx  unit  radiua  in  Figure  4  la  meant  to  represent  a  homo¬ 
genous  target  area  that  is  to  be  given  uniform  fractional  coverage  by  rtexa- 
gonal  pattern.  Assume  that  the  19  aiming  points  shown  have  been  given  equal 
quotas  of  bombs  to  optimize  the  specified  coverage  for  the  central  portion  of 1 
the  target.  Obviously,  thle  whole  target  area  can  be  given  coverage  as  indi¬ 
cated  by  the  tabulation  by  enlarging  the  pattern  sufficiently.  What  is  needed 
here  to  complete  the  requirements  is  a  statement  limiting  wastage,  or  peri¬ 
pheral  coverage  loss. 

2.2.1  The  art  of  tailoring  aiming  point  patterns  for  maximum  economy 
in  obtaining  a  desired  coverage  must  be  based  on  verifiable  principles.  One 
such  principle  deserving  consideration  is  an  existence  theorem  such  esi 

EXISTENCE  THEOREM,  For  every  distinct  target  area  with  specified 
•fractionaf coverage  by  a  definite  weapon  system,  an  optimum  pattern  of  aim¬ 
ing  points  exists,  and  a  discrete  set  of  isohap  curves  can  be  constructed  with 
sufficient  accuracy  to  moke  feasible  a  Monte  Carlo  method  of  predicting  the 
resulting  coverage. 

2.2.2  A  second  principle  deserving  consideration  for  verification  or 
rejection  has  to  do  with  the  progressive  modification  of  aiming  point  patterns 
associated  with  a  graduated  series  of  targets.  For  example,  a  target  that 
has  outgrown  a  single  aiming  point  must  accept  a,  three  point  pattern  if.  circu¬ 
lar  symmetry  is  to  be  even  approximately  maintained.  Thereafter,  the  next 
Bize  can  be  accomodated  by  four  points,  and  perhaps  by  five.  But  a  six 
point  pattern,  either  as  a  pentagon  wtlh  a  center  or  a  centerless  hexagon, 
may  have  to  give  way  to  seven  points.  Such  center  points  need  not  have  the 
same  allocation  of  bombs  as  the  others.  What  la  needed  is  more  .han  Just  a 
continuity  principle,  since  a  formulation  of  this  discrete  co.itlm.ity  la  also 
desirable.  The  principle  may  be  tentatively  stated  thus: 

CONTINUITY  THEOREM.  Targets  that  differ  slightly  in  size  and  shape, 
and  In  specified  coverage  by  a  particular  weapon  system,  will  have  aiming 
point  patterns  differing  moderately  In  configuration  and  bomb  allocation,  and 
Isohap  curves  bearing  strong  resemblances  In  all  characteristics. 

3.0  COMPUTATIONAL  SIMPLIFICATION. 


so 
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3.1  ARC  GOVgRAGE ,  In  paragraph  1.1.1  the  use  of  dividers  In 
occuimlsrtiog  arc  covers  go  of  *n  isohap  otarVs  was  shown.  Such  a, curve 
can  be  replaced  by  a  discrete  set  of  points,  thereby  allowing  the  simple 
act  of  counting  to  replace  continuous  arc  measurement.  Such  points  when 
hit  change  only,  on  attribute,  and  when  hit  a  second  time  retain  that  same 
attribute,  thus  making  unnecessary  any  special  consideration  of  overlap. 

If  a  rncro  reollcttc  traatmfent  of  tho  damaging  effects  of  a  bomb  hit  era' 
desirrd,  on*  may  ensign  full  kill  to  an  approprUio  circular  ares  around 
the  center  of  impact,  and  diminishing  fractions  to  points  in  surrounding 
rings.  Tho  attribute. "hit*  would  give  way  to  the  accumulation  of  fractions 
at  each  point  on  an  isohap,  with  unity  representing  saturation  or  full  kill. 

4.  SUMMARY.  A  Sftlvo  of  bombs  with  constant  ’cthal  areas,  aimed 
at  a  single  point  with  circular  error  probability,  give  equal  hit  probability 
to  points  oquidlotar.t  from  the  aiming  point.  Advantage  U  taken  of  the  re¬ 
sultant  central  symmetry' --In  target  coverage  around  this  pr  int  In  building 
aiming  point  patterns  that  provide  tha  area  within  the  pattern  maximum 
uniformity  in  coverage.  Submitted  for  clinical  consideration  ere  suggestions 
for  modifying  tho  a  a  optimum  patterns  to  accomodate  Irregular  and  nonhomo-  ■ 
gehous  targets,  and  for  simple  techniques  In  evaluating  coven ge. 
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FOR  A  RECTANGULAR  GRID  OF  AIMING  POINTS 


.  AN  EXPERIMENT  IN  PERSONNEL-  MANAGEMENT  EVALUATION* 

l*  ST,  Blough 

StattottfiasS  Kaseerch  Center 
TTnlverrelty  of  Chicago 


BACKGROUND.  A  personnel  management  program  may  be  subdivided  on 
paper  Into  classes  with  auch  tit  lea  aa 

Recruitment  and  placement 
Job  claaaificatioh 

Incant lve a  and  awarda  - ~- 

Disciplinary  actiona. 


Each  claaa  may  be  further  broken  down  into  a  Hat  of  dutlea  or  actiona  by 
management  In  connection  with  employees.  Without  further  defining  the 
elementa  constituting  the  program,  we  might  ask  auch  questions  as, 

Does  auch  a  program  do  any  gpod,  or  any  harm,  and  If  ao,  how 

much? 

Which  possible  elements  of  a  program  should  be  retained  and 
which  discarded  In  order  to  achieve  maximum  benefit*  ? 

As  used  here,  MgoodH  might  mean  an  incraae*  in  productivity,  or  In  qual¬ 
ity  of  production,  or  In  employee  satisfaction. 

Not  much  la  known  about  answers  to  these  questions.  One  raaton 
may  be  that  few  organizations  are  large  enough  to  have  the  facilltlae  for 
finding  answers  experimentally.  In  1957,  however,  a  project  was  under¬ 
taken  by  the  Office  of  the  Deputy  Chief  of  Staff  for  Personnel  at  Hq  5th 
Army,  Chicago,  to  be  directed  by  Arthur  Barbour  and  Baldwin  Sears  of  that 
Office,  to  acquire  quantitative  information.  The  Statistical  Research  Cen¬ 
ter  was  consulted  in  connection  with  design  and  analysis.  It  seamed 
that  the  experimenters  in  thts  instance  at  least  had  adequate  manpower  and 


*Thls  paper  outlines  an  experiment  described  In  more  detail  in  SRC-600624- 
Bg38,  a  report  of  the  Statistical  Research  Center  dated  24  June  1960. 

This  work  was  sponsored  by  the  Army,  Navy  and  Air  Force  through  the  Joint 
Services  Advisory  Group  for  Research  Groups  In  Applied  Mathematics  and 
Statistics  by  Contract  No.  Nfiori -02035.  Reproduction  in  whole  or  In  part 
is  permitted  for  any  purpose  of  the  United  States  Government. 
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facilities  wr  expertinentff  and  replications:  300,000  civilian  employe*! 
of  the  Department  oS  the  too*  at  nunaeroua  installations  throughout  tbs 
country. 

In  their  initial  trial!,  tha  experimenter*  decided  to  te«t  the  be«t 
and  mont  comprehensive  personnel  management  program  they  could  devise, 
under  the  moat  favorable  conditions  for  observing  the  effects  of  the  pro- 
gram.  If  measurable  effects  indeed  resulted,  experiment*  would  then  be 

devised  to  investigate  the  program  elements  individually. 

The  maximum  opportunity  for  observing  improvement  might  appear 
to  exist  at  an  installation  having  no  formal  personnel  management  program 
at  ell  prior  to  the  start  of  the  experiment.  But  auch  a  primitive  situation 
Would  likely  exist  only  if  the  local  commandant  or  management  were  Un¬ 
sympathetic  toward  personnel  management  programs  and  ao  probably  toward 
the  proposed  experiment.  In  any  casa,  such  an  installation  would  lack 
the  trained  personnel  specialist*  capable  of  performing  the  experiment.. 

As  a  compromise  it  woj  necessary  to  choose  an  installation  having  a  rea*on- 
ably  good  personnel  management  program  already,  expanding  thia  program 
to  "optimal-  for  the  experiment; 

The  Decatur  Signal  Depot,  Decatur,  Illinola,  which  waa  choaan 
.  for  tha  initial  investigation*,  had  a  management  personnel  program  “level" 
rated  by  the  experimenter*  a*  70  per  cent  of  optimal.  Thus  only  the  effect 
of  raising  the  level  from  70  per  cent  to  100  per  cent  could  result,  and  thia 
effect  might  not  be  large.  However,  even  a  small  effect  might  be  well 
worth  achieving.  The  experimenters  estimated  the  equivalent  in  annual 
wages  of  a  5  per  cent  productivity  increase  throughout  the  Department  of 
the  Army  to  be  about  $75,000,000. 

The  variable  of  most  Interest  at  this  time  was  in  fact  productivity, 
so  the  experiment  was  designed  on  that  basis.  Employee  satisfaction, 
of  which  typical  Indicators  are  assumed  to  be  so-called  "employee  reac¬ 
tions"— sick  leave  use,  injuries,  AV/OL,  voluntary  separations,  suggestion! 
—was  to  be  looked  at  incidentally,  with  interest  In  possible  correlation 
with  productivity.  Quality  of  produ.*  -  too  subjective  to  be  reliably 
assessed,  and  was  —  o*  -  •arisble  of  much  concern  in  the  experiment. 

DESIGN.  The  de-stgn  envisaged  a  minimum  of  15  independent  employee 
groups  already  existing  in  the  organization,  of  slzo  at  least  10,  and  aa 
alike  as  possible.  (A  primary  objective  here  was  to  provide  good  condition* 
for  observing  an  effect  If  present.)  All  groups  should  already  hove  in  routine 
operation  a  procedure  for  measuring  productivity.  The  groups  would  be 
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assigned  randomly  to  3  categories  corresponding  to  what  were  familiarly 
called  treatments  (Table  11. 

Table  1 


These  groups  ere  to  be 
Informed  of  the  experiment 
but  otherwise  will  remain 
under  the  usual  conditions. 

3.  Experimental  groups  The  personnel  management 

program  applied  to  these 
groups  Is  to  be  Increased 
to  100  per  cent  of  optimum. 

(The  Informed  controls  were  Included  to  provide  against  and  test  for  the 
so-called  Hawthorne  effect,  the  effect  on  the  subjects  of  merely  being 
part  of  an  experiment.)  Monthly  data  were  to  be  collected  for  1  year,  or 
some  other  suitable  lengthy  period,  before  the  actual  start  of  the  experts 
ment.  The  treatments  would  then  be  started  and  data  collected  for  a 
comparable  period  during  application  of  the  treatments.  The  analysis 
was  to  be  performed  on  numbers  representing,  for  each  group,  the  ratio 

treatment  period  performance 
pretreatment  period  performance 


Category  • 

1.  Uninformed  controls 


2.  Informed  controls 


Treatment  to  be  applied 
to  groups  In  category 

No  treatment  at  all.  It 
Is  assumed  that  these 
groups  operate  under  the 
usual  conditions  and  are 
ignorant  of  the  experiment. 


IMPLEMENTATION.  The  actual  experimental  setup  fell  somewhat  short  of 
the  specifications.  14  employee  groups  were  originally  chosen  for  the 
experiment,  of  which  5  were  later  dropped,  leaving  9  groups  (3  per  treat¬ 
ment)  Instead  of  the  recommended  minimum  of  IS  groups.  The  assumption 
of  Independence  lor  these  groups  appeared  reasonable,  but  their  sizes 
ranged  from  4  to  19  employees,  and  they  differed  in  composition  ( 2  wars 
partly  made  up  of  women).  While  most  groups  worked  at  storage  and 
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handling  of  various  types  of  electronic  equipment,  one  did  clerical  work 
and  one  manufacturing.  Further,  productivity  standards  ware  applicable 
to  only  about  50  to  75  per  cent  of  the  groups'  jobs,  so  measurements  of 
productivity,  reflected  only  a  fraction  of  each  group's  total  work*  Data 
wore  available  for  only  4  months,  Nov  'S71  -  Feb  '■58,  prior  to  the  start 
of  the  treatments,  and  for  16  months  after  start  of  the  treatments.  Mar  '58 
-  Jun  '59.  During  the  latter  "treatment  period"  the  experimenters  esti¬ 
mated  that  the  personnel  management  program  level  for  the  "Experimental 
groups"  rose  rather  gradually  from  70  per  cent,  reaching  "very  nearly" 

ICO  per  cent  during  the  period  Oct  *58  -  Feb  *59  end  then  falling  off. 

ThuB  the  experiment  proceeded  under  a  number  of  handicaps  which  had 
not  been  foreseen. 

PATft.  The  productivity  data  were  constructed  as  follows.  Suppose  for 
Job  V  a  time  study  has  specified  HJ/  hours  per  unit  of  production. 

If  an  employee  actually  spends  Ay  hours  producing  Uy  units,  the 
product  HyUy  Is  called  the  "Earned  Hours"  and  Ay  the  "Actual 
Hours"  for  that  amount  of  work,  and  the  corresponding  productivity  Is 

100  BL3J  .  100  EfliMlEffllli  , 

Ay  Actual  hours 

a  measure  of  the  productive  use  of  time.  Total  productivity  for  work 
done  on,  say,  n  Joba,  la 

n  n 

100  Hy^V  /  Jl  Ay’.  Once  HyUy  and  Ay  are  ob¬ 

tained  for  each  employee's  work  on  each  Job  each  day,  productivity  for 
any  combination  of  employees,  jobs,  end  days  may  be  calculated.  In 
the  routine  collection  of  productivity  data  at  Decatur,  Earned  end  Actual 
hours  were  respectively  summed  within  each  group  over  an  entire  month, 
bo  the  ratio  reported  for  each  group,  each  month,  was  a  monthly  produc¬ 
tivity  for  the  group,  applying  oi  course,  only  to  that  part  of  the  group's 
work  covered  by  the  time  study  standards, 

V 

Data  on  the  above  mentioned  employee  reactions  (considered 
indications  oi  employee  satisfaction)  were  reported  .in  terms  of  Index 
numbers  apparently  Intended  to  express  the  various  "reactions"  in  com¬ 
parable  units,  as,  say,  percentage  of  a  "norm",  where  the  norm  Is  usually 
some  average  of  past  experience.  It  Is  not  clear  that  the  index  numbers 
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art  always  more  Informative  or  easier  to  interpret  than  the  actual  observed 
quantities.  For  example,  the  index  originally  adopted  by  the  experiment 
ters  for  use  in  this  study  for  reporting  reactions  whose  Increased  frequency 
of  occurrence  indicates  a  decrease  in  employee  satisfaction  (s.g.,  sick 
leave  usage)  was  computed  as 

(Norn  -  Observed)  *  100  •  Index, 


for  example, 
or 


(93  -  100)  *  100  -  08, 
(.5  -  2.5)  4  100  «  98. 


The  index#*  finally  adopted  for  the  experiment  were,  for  such 
"desirable"  employee  reactions  as  "suggestion**,  • 

i°o  ygi- , 


end  few  "undesirable”  reactions, 


ioo  f  i  -  . } .  m  1 2  -  Mau, ) . 

'  Norm  /  V  Norm  1 


(These  latter  Indexes  are  now  standard  for  reporting  about  a 
hundred  different  items  in  routine  evaluations  of  the  Army's  Civilian 
Personnel  Program.  However,  the  universal  usefulness  of  the  Indexes 
Is  not  clear,  as  Illustrated  by  an  example  arising  in  the  present  experi¬ 
ment.  A  Norm  for  voluntary  separations— mainly,  number  of  employees 
quitting— was  computed  as  the  average  percentage  of  the  wcric  force 
being  separated  voluntarily  per  month  for  each  month,  from  data  collected 
over  an  earlier  2 -year  period.  There  were  2  voluntary  separations  from 
all  9  groups  over  the  20  month  period  of  the  experiment,  one  of  these 
occurring  in  a  group  of  size  22,  for  a  rate  of  4.545  per  cent  in  the  month. 
The  norm  for  that  month  was  .247  per  cent,  and  resulting  Index  was 
100(2  -  4. 545/. 247)  »  -1640— representing  the  smallest  nonzero  separa¬ 
tion  rate  that  could  have  been  obtained  for  this  group— to  be  compared 
with  the  index  200  for  a  zero  rate.) 


Because  of  their  rather  dubious  meaning,  the  index  numbers  were 
not  used  in  the  analysis  of  the  experiment.  Actually,  for  the  most  part, 
there  were  too  few  occurrences  among  the  "employee  reactions"  to  permit 
analysis. 
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ANALYSTS.  In  general,  let 

1*1,  2,  I  treatment*.  In  this  case  1*  3. 

J  ■  1,  2,  . . J  groups  within  each  treatment.'-.  Here  j  •  3. 
k*l,  2,  ...,  K,p  pretreatment  month*.  Here  JCy-4. 

e 

k'  ■  1,  2,  .  ..,  Kj'  treatment  months.  Here  *  16. 

Then,  for  group  J  within  treatment  1,  and  prstraatment  month-  k». 
designate  the  productivity  random  variable  by  Xji j)<>  and  let  be 
the  random  variable  donating  thus  same  group's  productivity  in  treatment 
month  k'.  Milting  the  transformation  to  log  and  log  Yjj^t  (in 
ordur  to  normaHr-e  the  productivities',  which  are  ratios)  and  averaging, 
for  group  J  within  treatment  l,  the  tranuformed  pretreatment  period  data 
and  treatment  period  data  over  desired  sets  of  months,  form  the  difference 

K2  K} 

•  £  lo»  vi)k*  -  kT  -  I  109  xijk; 

1  k-  k 

where  Kj  *  some  number  of  pretreatment  months,  0£lCj£Kpj 
^  *  some  number  of  treatment  months,,  0  **2<Kr 

Then 

Z  -  jog  ( geometric  mean  of  K;  treatment  monthly  productivities  \ 

^  \  geometric  mean  of  K4  presentment  monthly  productivities  / 


the  geometric  means  being  a  consequence  of  the  log  transformation.  The 
operations  of  forming  Z..  may  be  assumed  to  have  removed  the  group 
effect,  and  since  groups'ere  independent  of  each  other,  to  have  resulted 
In  an  observation  for  each  group  to  which  the  following  simple  model  . 
applies: 

ziym  (A  +  & i  +  «ij 


where 

J1  -  over-all  mean 

Of  ■  treatment  effect,  E«t  -  0. 
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•  *>  random  error,  normally  distributed  with 
men  0  and  variance  cr,  the  ■•y*t 
being  mutually  Independent. 

Analyses  may  now  be  performed  on  observed  values  of  the  Zy'a  calcu¬ 
lated  from  the  productivity  data. 

The  Zji'a  could  be  formed  from  averages  over  any  months  avail¬ 
able,  and  In  fact  5  analyses  of  productivities  were  performed  ustngverlous 
combinations  of  monthly  observations.  The  analysis  which  employed  pro¬ 
ductivity  comparison  ratios  for  the  two  periods  Nov  *57  -  Feb  '58  and 
Nov  *59  -  Feb  ‘59  appeared  to  be  the  most  appropriate  and  Indeed  gave  the 
lowest  estimate  of  residual  variance.  The  results  of  this  analysis  are 
summarised  below  In  Tables  2  and  3.  Table  2  contain!  point  estimates 
pfii&i  of  /44C?  i)  converted  back  (by  taking  antilogs)  to  a  (treatment 
perlcd)/{pratreatmont  period)  productivity  ratio.  Table  3  contains  antilogs 
of  differences  (/*♦&)  -  t  that  is,  the  entries  are  ratios  of  the 

ratios  in  Table  2,  and  of  95  per  cent  confidence  limits  for  the  true  differ¬ 
ences,  also  converted  back  to  retloa. 


Table  2 


1 


1 

Uninformed  controls 

1.06 

2 

Informed  controls 

1.02 

3 

Experimental  group* 

.98 

3/2  Experimental/lnfonned 

2/1  informed/Unlnformed 

3/2  Ixperlmentel/Unlnformed 


.92 


.80,  1.14 
.81,  1. 14 
.77,  1.10 


is 
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The  estimates  of  Tftblet  I  say,  for  example,  fflhMt  the  Irtrionwd 
controls  improved  6  per  c*ott  derta®  ttae  treatment  period  as  compared  to 
their  performance  In  the  pretreBKaueat  period,  while  the  Experimental 
groups  declined  2  per  cent.  In  Table  3  the  relative  Improvement  of  the 
Experimental  as  compered  with  the  Informed  groups  was  .96,  which  super¬ 
ficially  suggests  that  optimization  of  the  personnel  management  program 
is  detrimental.  However,  the  confidence  limits  for  the  ratios  of  Table  3 
obviously  Indicate  such  large  variability,  that  nothing  Is,  and  little  could 
be,  statistically  significant.  In  fact,  the  power  of  this  teat  against  a 
real  5  per  cent  Increase  in  productivity  was  estimated  as  about  .1,  and 
to  raise  the  power  to  .9  would  require  an  estimated  44  groups  per  treat¬ 
ment,  or  132  groups  in  all,  a  seemingly  prohibitive  number.  . 

The  sketchy  analyses  of  employee  reactions  which  were  poesible 
also  showed  no  statistically  significant  effects  which  could  be  attributed 
•  to  the  treatments.  One  of  the  groups  did  Bhow  a  statistically  significantly 
higher  rats  of  sick  leave  usage  than  the  others.  This  group  was  small, 
with  a  high  proportion  of  women  employees.. 

SOURCES  OF  ERROR.  Some  possible  contributors  to  the  large  variability 
are 

1.  Differences  between  groups  In  composition,  slzs,  and  type 
of  work. 

2.  Supervisory  differences  and  differences  in  this  personnel 
management  treatment  received  by  groups  within  a  given  category. 

3.  Differences  created  by  the  standards  of  productivity.  For  ex¬ 
ample,  time  study  may  allot  too  few  hours  (say  Hs)  or  too  many  hours 
(Hy )  per  unit  of  production.  Then  for  A  hours  actually  spent  prbduclng 
U  units, 

HU  H.U 

— — -  c.  — —  * 

A  A 

That  is,  the  apparent  productivity  depends  on  the  standards,  and  shifting 
from  jobs  with  strict  standards  (H#)  to  those  with  tenient  standards  (H£) 
will  cause  an  apparent  increase  in  productivity  when  the  actual  producti¬ 
vity  is  unchanged.  Standards  for  the  same  lob  are  often  revised,  but  this  1 
is  not  believed  to  have  happened  during  this  particular  experiment. 
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TtesU^ofc'EcpettirnertUi 

Also,  standards  nsf  be  east  up,  in  installation  like  Decntor,  to  •pAf 
to  handling  individual  items.  Occasionally,. large  orders  will  require 
handling  of  gross  lots  by  lift  truck  with  consequent  remarkable  temporary 
rises  in  reported  productivity. 

4.  Fluctuating  workload.  For  example,  during  this  experiment 
the  invasion  Of  Lebanon  occurred,  which  caused  a  great  Increase  In  de¬ 
mands  made  on  this  installation. 

5.  Errors  in  collecting,  computing  and  reporting.  The  task  of 

recording  and  computing  and  A^  for  all  employee  x  day  x  Job 

combinations  c  jntains  many  opportunities  for  error.  In  one  case  (found 
in  previous  work  where  raw  data  were  examined  in  detail)  one  employee 
on  one  day  on  one  Job  was  reported  to  have  produced  1403  units.  His 
productivity  was  1822.  The  Job  number  turned  out  to  be  1403,  and  this 
apparent  error,  when  eliminated,  reduced  the  group’s  monthly  figure 
from  106  to  102.  Thus  a  single  error  had  increased  the  group's  reported 
productivity  by  an  amount  comparable  with  that  of  the  effect  looked  for 
in  this  experiment. 

Other  basic  difficulties  may  be  inferred  from  the  fact  that  after 
close  of  the  experiment  the  experimenters  said  that  they  doubted  that 
the  optimization  of  the  personnel  management  program  sought  for  tha 
Experimental  groups  had  been  attained,  and  that  the  level  which  the  . 
treatment  had  actually  reached  was  not  very  precisely  known. 

COMMENTS,  Improvements  in  experimental  technique  are  evidently  re-  • 
quired  to  obtain  useful  results  from  future  experiments  of  practical  size. 
It  is  likely  that  variability  can  be  reduced  by  such  means  as  care  in 
selecting  groups,  elimination  of  clerical  errors,  and  exclusion  of  data 
arising  from  abnormal  circumstances.  However,  t  appears  only  prudent 
at  this  stage  to  utilize  as  many  employee  groups  as  possible  to  attempt 
to  overcome  the  effect  of  variability  still  present. 

There  remain  the  basic  requirements,  such  as,  that  the  groups 
must  be  and  must  remain  Independent,  and  must  receive  the  treatment 
specified.  It  would  seem  that  only  local  management  can  assure  that 
even  the  most  general  design  conditions  are  met,  and  so,  as  essential 
participants,  local  management  should  have  adequate  understanding  of 
the  experiment  and  its  objectives. 


A.  NOTE  on: APPROXIMATE  CONFIDENCE  INTERVALS  FOE  FUNCTIONS 

«E’iDKi!mi.T8a*Mmaa 


Henry  DeClcco  . 

Ordnance  Special  Weapons  -Ammunition  Command 
Dover,  New  Jersey  . 

I.  INTRODUCTION.  A  system  Is  made  up  of  a  number  of  components  in 
arbitrary  combination,  and  it  la  required  to  obtain  a  confidence  Interval 
for  the  reliability  o!  the  system  without  testing  the  whole  system  itself. 

That  la,  we  have  at  our  disposal  only  the  teat  data  (let  ua  asuume  In  the 
form  of  binomial  success  ratios)  on  the  components  of  the  system. 

Special  cases  of  this  problem  have  been  treated  by  Buehler  (1)  and 
Madansky  (4).  The  method  presented  herein  leads  to  approximate  confidence 
Intervals  but  is  general  enough  to  cover  arbitrary  systems  with  relative 
ease,  It  is  also  capable  of  accommodating  the  case  where  the  components 
of  the  system  are  statistically  dependent,  although  this  case  is  not  devel¬ 
oped  here.  The  method  involves  computing  moments  of  functions  of  random 
variables,  in  particular  those  functions  of  the  observed  binomial  data  des¬ 
cribed  by  the  probability  structure  of  systems  of  Interest.  Although  It  Is 
entirely  feasible  to  compute  the  firet  four  moments  of  such  functions  and 
thereby  settle  the  question  of  a  relevant  distribution  function,  practical 
work  generally  requires  no  more  than  the  first  two.  The  following  discussion 
Is, accordingly,  so  limited. 

n.  N  INDEPENDENT  COMPONENTS  IN  SERIES.  Eat  ft  denote  the  observed 
k  successes  in  m  binomial  tests  recorded  for  the  1-th  component,  and  let 
Pl  denote  the  associated  binomial  parameter.  Fora  system  of  n  statistically 
Independent  components  in  series  we  have 

C2.D  F(pl#p2,  ...$„) -JJpj  1-1,2,. ..n 

(2.2)  £(F)  -  *ffp| 

(2.3)  Var  (F)  "  1|f  (P i  4  Var  pj)  -  Tfp] 

the  last  two  following  directly  from  the  definitions  of  expected  value  and 
variance. 

A  more  explicit  form  of  (2.3)  for  computation  is 
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J V|L.  miyUMUKW  w 

11.3.)  v»rtn-  Z  v.rp1?  4  - 

•  f3(J.2,3"2.)v*r^v,rV 

-v  I  .  (1.  PH2  )  Var  Var  pt  . .  .Ver  pt 


2  3 


U  Var  pi 


wh*r*  ell  distinct  subscripts  are  summad  from  1  to  n  to  yiild  2n  -1  taraa 
and  where 

•  .  S.  . 

Ver  pt  *  pj(l  -  Pi)/m1  . 

While  the  last  result  is  exact,  It  is  easily  seen  that  a  serviceable 
approbation  exists  In  lta  linearized  version,  obtainable  directly  from  the 
classical  propagation  of  error  formula,  that  la 


(2.4)  Var  (F)«  *- 

1 

where  the  partlals  are  understood  to  be  evaluated  at  tha  point  p.,  Pj, 

....  p  .  Applying  this  to  (2.1)  gives 
n 

(2.5)  Var(F)^  Z  (  .  ]T  P? 

*2  h^l2  *1 


which  Is  Just  the  first  n  terms  out  of  the  total  2n  -  1  in  (2,3a). 
Application  of  (2.4)  naturally  assumes  that  F  has  been  redefined  for  con¬ 
tinuity  since  the  ^‘s  take  on  only  fractional  values  or.  the  unit  Interval. 

Finally,  corresponding  to  (2.2)  and  (2.3)  the  relevant  unbiased 
estimates  are  cosily  shown  to  be 


PI  2  0  -  Pl2> 
"12 


n 


IR.  MORS  GENERAL  SYSTEMS.  Tha  probability  structure  of  each  system 
Is  of  course  special,  and  It  would  be  pointless  to  attempt  a  catalogue  of 
these.  Sven  so,  tt  might  be  useful  to  characterize  a  fairly  general  etruc- 
ture  to  suggest  the  flexibility  Of  the  method  of  linearized  estimates.  Such 
a  structure  might  be  as  follows:  •  assuming  statistical  Independence  through 
out,  we  consider  n  assemblies  In  series  where  an  assembly  Is  made  up  of 
Sj  Identical  components  in  parallel  and  where,  further,  at  least  at  of  the 
Bj  components  must  function  for  the  assembly  to  function.  We  then  have 
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(2.6)  t  CD  -  |fp4 

(2.7)  Var  (F)  -  ^  p*  - 


(3.1)  F 


TFfi 

U"®!  J 


1-1.  2, 


and  to  «  first  order  approximation 


(3.2)  E(T)^^j 


(3.3)  Var  (F)  sa 


i  (  if  l  £  n  *P»  )*‘‘i  • 

l2  |ll^l2  Ny“alj  ' y '  l|  ll  / 

(yt  (y‘J)  [' V‘ “‘S1  -V>%,8V‘2,rl| 


^  A 

Var  p. 
l2 
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■where,  as  befo?e ,  bsrth  subscripts  tram  Manned  from  t  %e  nu  Tin  foregoing 
are  usually  biased  in  keeping  with  a  general  limitation  of  linearised  esti¬ 
mates..  (Refer  Concluding  Remarks. 1 

In  the  case  of  simply  redundant  systems  where  Bj  «  1  we  readily 
obtain  the  corresponding  expressions 

(3.1a)  F-  IJ'  {l -  0-p/1} 

(3.2a)  E  CP)  ^Tfl  - 

(3.3a)  Var  (F)£  Z  'J  If  ft-[l  -P^]  *l2  Cl-Pij)  i%  j  Var 

1^2 

IV.  APPROXIMATE  CONFIDENCE  INTERVALS.  The  limited  experience  of 
the  writer  to  date  with  systems  of  particular  interest  to  Ordnance  has  indi¬ 
cated  that  Var  (F)  is  generally  very  small  compared  to  £  (F).  We  now 
quahtyy:the  term  "small"  to  determine  a  numerical  condition  under  which  the 
first  two  moments  as  discussed  above  are-enough  to  give  reasonably  useful 

confidence  intervals.  To  do  this  we  examine  the  ratio  E  (F)  /  VVar  F  in  the 
context  of  fcheby cheff ' s  inequality.  We  readily  obtain 


(4.1) 


F  -  E  (F) 
E  (F) 


1  - 


1  Var  (F) 
f2  E2  (F) 


Which  has  the  common  sense  interpretation  that  the  larger  the  ratio 

E  (F)  /Yvar  (F),  the  smaller  is  the  probability  that  a  particular  observation 

of  our  chance  quantity  F  (p\,  pj,  ...  Pg)  will  deviate  beyond  a  given  dis¬ 
tanced  from  E  (F). 

Figure  1  is  a  plot  of  the  bound,  £  of  the  relative  deviation  against 
ttve  ratio  E  (F)  /"\)Var  (f)  with  confidence  level  &  as  parameter.  'That  is, 
the  condition 
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Implies  the  relation 

(4.2)  €  .VVf-JB. 

■Jft w. 

In  Figure  l,  S  is  taken  a*  .10  for  the  90%  confidence  Interval  usually 
desired  In  engineering  applications. 

The  ratio  E(F)/Vvar  (t)  la  commonly  referred  toes  the  "slgnal-to- 
nolse  ratio."  Designating  this  ratio  by  R,  we  recognize  the  plot  of  Figure 
1  as  hyperbolic;,  that  is, 

t  -  k/R,  k  «  1/VTT 

We  see  that  when  the  ratio  exceeds  30,  roughly,  the  relative  devia¬ 
tion  is  not  likely  to  exceed  10%.  This  Is,  of  course,  saying  nothing  more 
than  that  some  3  standard  deviations  on  either  side  of  the  mean  value  of 
a  distribution  (unspecified  save  for  having  a  finite  variance)  will  eoverabout 
90%  of  the  range  of  values.  However,  the  real  advantage  of  such  a  plot  Is 
that  it  shows  that  after  a  certain  petnt,  large  values  of  E  (F)/V  Var  (F) 
do  not  influence  the  bound  6  very  much.  The  fact  that  the  curve  in  Figure  1 
is  relatively  flat  over  a  whole  region  is  often  useful  In  deciding  when  esti¬ 
mates  of  even  the  first  two  moments  are  enough  to  settle,  in  a  practical 
sense,  questions  concerning  whether  a  prescribed  level  of  reliability  for  a 
complicated  system  is  likely  to  have  been  satisfied.  Observe  further  that 
this  fact  also  allows  for  considerable  imprecision  in  the  estimates  of  both 
E(F)  and  Var  (F).  If,  after  such  a  computation  is  made,  one  requites  fuller 
Information,  it  would  be  necessary  to  calculate  higher  moments.  The  work 
of  Tukey  (5)  provides  expressions  for  the  first  fourmoments  that  go  consid¬ 
erably  beyond  the  level  of  refinement  of  linearized  estimates.  However,  a 
major  conclusion  of  that  work  is  that,  with  particular  reference  to  tho  classi¬ 
cal  propagation  of  error  formula  (that  Is,  the  formula  for  Var  (F))p  linearized 
estimates  are  often  better  than  commonly  supposed. 

It  might  be  well  to  emphasize  that  the  explicit  form  exhibited  by  lin¬ 
earized  estimates  of  Var  (F)  serves  the  further  useful  purpose  of  exposing 
those  components  and  substructures  of  a  system  that  appear  as  major  contri¬ 
butors  to  the  overall  variability.  A  rational  allocation  of  additional  compo¬ 
nent  tests,  for  the  purpose  of  reducing  that  variability,  is  thereby  Indicated. 
{.See  example  below.) 
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Ftoally,  a  ©Dnrpfcrtson  of  certain  c&Iculatlcms  given  by  both  Byehler 
0)  and  Madansky  (4)  indicates  good  agreement  With  the  method  of  this  note. 
Tor  example,  Madansky  (4)  gives  the  following  compart  son  with  a  result  of 
Bushier  for  the  upper  limit  of  a  9C%  confidence  interval  for  the  probability  of 
failure  of  a  two  component,  parallel  system,  where  3  failures  in  100  tests 
were  recorded  for  one  component  and  5  failures  in  100  tests  were  recorded 
for  the  other;  Bushier  (1)  obtains,  for  the  upper  limit,  .004.2.  Madansky 
(4)  obtains  .00518. 

Interpreting  -  3/100,  p2  *  5/100  now  as  failure  rates .  and  using 
equations  (2~S)  and  (2.7),  we  obtain  the  following  unbiased  estimates 

E  IF)  *  pjp,  -  .00150 

Var  CD  *  P*  P2  -  fpj  “/p^l 


-8 

-  102.56  x  10 

so  that 

E' CF)  +  3.162  ^Var  (F)  -.00470. 

The  corresponding  linearized  estimate  based  on  (2.4)  yields  an  upper  limit 
of  .00491  so  that  a  positive  bias  in  the  amount  .00021  is  thereby  incurred. 

v*  SAMPLE,  The  following  example  indicates  a  simple  application  of  (3.2a) 
and  (3.3a)  to  a  system  of  a  common  generic  type.  In  addition  to  computing 
a  lower  90%  confidence  limit  for  the  reliability  of  the  system,  we  exhibit 
the  structure  or  the  ’associated  variability  explicitly  and  also  take  note  of 
the  signal-to-noise  ratio. 

The  system  is  made  up  of  four  assemblies  in  series  (we  shall  assume 
statistical  independence  throughout).  The  first  assembly  consists  of  a 
single  component.  The  second  assembly  consists  of  two  identical  compo¬ 
nents  In  parallel,  at  least  one  of  which  must  function  for  the  assembly  to 
function.  The  third  assembly  consists  of  three  identical  components  at 
least  one  of  which  must  function.  The  fourth  assembly  consists  of  a  single 
component. 
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The  following  test  data  applies  faBt&anmd  success  satios): 

ft  ;  -  128 

1  200 


^  -  124 

*  200 


ft'  -  128 

J  200 

ft;  -  m. 

*  200 


Corresponding  to  (3.2a)  end  (3.3a),  respectively,  we  have 

E  (F)  •#  |  1  -  (l-ft/J  £  l  -  P4  *-968 

Var  (F)  ~  j^l-  d-ft/}  [  l-  U-^3)3J  ft4]a#  ftjft-fr) 

+  [2^  (l-if2)  £l  -  (l-$3)3]  ft4|;  g2(1~P2> 


198 

4 


2 

+[3Pj  fi*  d"^)2]  (*-ft3)2  p41 

*199 

♦  [ftt  (i  -  u-p2)2]  (i-  a -p3)3J ]  # 

199 

-« 

#  144.5  x  10  . 

We  obtain  the  desired  lower  limit 

Z  (F)  -  3. 16 War  (F)  -  .931. 
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The  components  of  the  variability  are  as  follows; 

-  47.68  x  ID'* 


«  ,4954  x  10  A 


-  .01262  x  10 “6 


-  93.33  x  10 


We  thereby  observe  that  some  two-thirds  of  the  total  variability  comes 
from  the  final  assembly  alone,  the  remainder  arising  almost  entirely  from 
the  first.  Observe  that  this  conclusion  is  far  from  obvious,  since  the 
assemblies  cited  are  precisely  those  with  the  lowest  observed  failure 
rates  and,  Indeed,  the  lowest  individual  variances.  It  is  therefore  cleaT 
that  the  most  direct  approach  to  reducing  the  overall  variability  would  be 
to  Increase  the  number  of  tests  on  the  fourth  and  first  component  types. 

rinally,  we  observe  a  signal  "to  “noise  ratio  about  80  which, 
according  to  Figure  l,  is  within  a  relatively  flat  region  of  the  bound  £. 
Considerable  variation  In  the  estimate  pf  E  (F)  /yfi aFlrT  therefore 

not  likely  to  influence  a  practical  decision  based  on  the  estimated  relia- 
blllty  of  the  system. 

VI.  CONCLUDING  REMARKS.  It  is  clear  that  if  the  observed  success 
ratios  on  all  the  components  of  a  system  are  either  aero  or  one,  then  the 
computed  variance  of  F  ($j,  f,,  ...  pn)  will  vanish  and  no  very  useful 
information  is  obtained.  Thlr  case  would  represent  an  intrinsic  limitation 
of  the  method  of  moments,  but  from  the  standpoint  of  applications  it  does 
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not  appear  to  be  one  fre^seatly  or  even  occasionally  encountered. 

The  bias  associated  with  estimates  of  E  00  and  Var  (F)  has  been 
considered  here  only  with  respect  to  series  systems.  It  la  possible,  In 
principle,  to  assess  and  remove  the  bias  implied  in  linearized  estimates', 
computed  for  more  general  systems ,  in  terms  of  expressions  for  moments 
of  the  kind  given  In  reference  (4).  The  amount  of  work  involved  in  this 
will  generally  be  prohibitive,  though  for  common  system  types  It  will  be 
feasible. 


ACKNOVrLEDG^MEMT.  For  discussions  of  the  problem  of  this  note  and 
its  ramifications,  the  writer  gratefully  thanks  Dl  H.  Evans,  A.  Stein, 

F.  E.  Grubbs  and  S.  Ehrenfeld. 
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The  objective  of  this  paper  Is  to  show  the  functions  which  describe 
the  relations  existing  between  static  teat  results  (average  motor  pressure 
and  50%  burning  time)  and  flight  test  results  (burning  distance,  burnt  velo¬ 
city,  maximum  cartridge  case  pressure,  muzzle  velocity,  and  bum-out 
time)  of  a  spin  stabilized  rocket..  A  second  objective  is  to  list  confidence 
limits  in  order  that  the  results  may  be  better  evaluated. 

The  equations  found  relating  flight  test  results  to  static  test  results 
are  generally  linear  or  of  linear  form,  except  for  one  logarithmic  term  due 
to  flight  temperatures.  Equations  predicting  burning  distance,  burnt  velo¬ 
city  and  burn-out  time  are  described.  These  equations  express  the  flight 
test  values  as  functions  of  various  static  test  results.  The  equations  which 
express  flight  test  values  are  given  in  terms  of  (i)  six  static  test  values, 

(ii)  four  static  test  values,  (iii)  three  values  found  from  the  six  static  test 
values,  and  (lv)  logarithms  of  six  static  test  values.  Generally,  the  equa¬ 
tions  which  involve  allthe  static  values  are  best  for  predictive  purposes. 

All  the  different  type  equations  mentioned  are  related  to  arithmetic  means  -  _* 
of  flight  test  results  and  to  arithmetic  means  of  static  test  results. 

Some  relations  similar  to  those  mentioned  above  were  found  involving 
logarithms  of  variances  rather  than  means.  These  equations  do  not  predict 
variances  with  as  much  accuracy  as  the  corresponding  eq”atlons  predict 
means.  Thus,  most  of  this  paper  will  be  devoted  to  a  discussion  of  the  best 
equations  which  use  means  for  variables. 

l’ 

e 

There  are  a  number  of  flight  variables  but  of  the  three  flight  variables, 
burning  distance,  burn-out  time,  and  burnt  velocity,  the  latter  can  be  pre¬ 
dicted  with  the  most  accuracy.  The  best  equations  predicting  burnt  velocity 
are  as  follows: 

y,  =  -5603.6  -  66004X  -  .69340X  4  .32081X  -  589.54X. 

«  1  2  3  4  ■ 

-1359. 2X3  4  733.30Xg  +  3675.9  log  (X?  4  623),  (1) 
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gives  the  relation  between  burnt  velocity.,  yb,  in  feet  per  second  and  the 
six  static  variables  and  temperature,  where  Xj,  Xj,  represent  average 
motor  pressures  In  pounds  per  square  ineh  at  -40°F,  70®P,  and  160OF,  re¬ 
spectively;  X4,  Xj,  Xg  represent  50%  burning  time  In  eeconds  at  -4Q°F, 

70°F,  160°?,  respectively;  and  X7  Is  one  of  the  flight  temperatures  -20°F, 
70°F,  140°F.  .  ■ 

yb-  1352  +  .07765X2  +  .1536X3  ♦  543. 2XS 

-  268. 7Xg  +  2.329X7  (2) 

gives  burnt  velocity  In  terms  of  only  five  variables,  X2.  X3,  Xg,  Xg,  X7, 
defined  as  for  equation  (1). 

yb- -  7831.7  -  1.6075UJ  -  97. 430u2 

+  386. 49u3  +  3387.7  log  CX?  +  578.8)  (3) 

gives  burnt  velocity  as  a  function  of  four  variables  u^,  u^,  Ug,  andX?, 
where  u.  “log  X.  -  log  X.,  y,  ■  log  X.  -  log  X  ,  U-  ■  log  X  -  log  X  ,  and 
X?  is  defined  as  above. 

All  three  of  these  equations  are  highly  predictive.  Just  how  good 
equation  (1)  13  as  a  predictor  of  the  actual  flight  values  can  be  seen  by 
looking  at  Tables  I,  11,  and  III,  where  it  will  be  observed  that  the  predict¬ 
ed  values  are  less  than  4.5%  in  error  when  compared  with  the  actual  flight 
values.  The  99%  confidence  intervals  for  predicted  values  at  -20°F,  70OF, 
140°F  are  respectively, 

2461. 6£  y£  2593.4, 

2686.6  £y<  2815.4, 

2840.8  ^y  2969.6. 
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predicted  Plisrht  Measurement!  Uaing  Equation  (D 


Mix  - - 

Number  Actual 

Value 


1495 

1157 

U74 

1187 

1248 

1261 

1280 

1293 

1308 

1342 

1351 

1399 

1361 

1372 

»373 

1385 


2440 

2588 

2580 

2628 

2550 

2502 

2480 

2528 

2482 

2497 

2500 

2555 

2558 

2496 

2475 

2472 


at  -20°F 


Predicted 

Value 

2486 

2529 

2531 

2513 

2560 

2529 

2469 

253B 

2525 

2516 

2490 

2523 

2490 

2534 

2541 

2563 


Percent 

Value 


1.80 
-2.27 
-1.89 
-4.37 
.39 
.99 
-  .44 
.39 
1.73 
.76 
-  40 
-1.25 
-2.65 
1.52 
2;66 
3.68 


Number 


Actual 

Value 


2837 

2954 

2893 

2980 

2952 

2890 

2878 

2910- 

2950 

2932 

2764 

2890 

2865 

2852 

2910 

2888 


Predicted 

Value 


Percent 

Value 


-3.05 
-,54. 
.51 
-1.14 
.  13 
-1.66 
-1.36 
3.69 
.31 
.03 
1.96 
.24 
1.76 


A  study  similar  to  the  relationship  between  static  firings  and  flight 
firings  have  been  considered  by  the  Army  Rocket  and  Guided  Missile  Agency 
involving  physical  test  data.  Propellants  designated  A,  B.  C,  and  D  were 
mixed  and  cast  in  cylindrical  specimens  and  subsequently  guillotine  sliced 
and  dog  bones  stamped  out.  Propellants  A,  B.  and  C  specimens  had  two 
vertically  aligned  dots  spaced  one  inch  apart,  and  a  photographic  technique 
was  used  to  measure  the  longitudinal  extension.  This  technique  should 
eliminate  minor  dimensional  changes.  Propellant  D  did  not  utilize  the  photo¬ 
graphic  technique.  In  all  cases,  however,  the  propellant  was  obtained  from 
a  cylindrical  carton  and  no  attempt  was  made  to  number  these  propellant 
samples  to  designate  them  from  adjacent  samples.  All  propellants  are  within 
batch  data  except  propellant  O  which  is  five  batches. 
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property  data  are  summarised  below: 


Propellant 


A  B  C 


1 

A 

O 

78 

143 

*40 

.  7B 

BSD 

-40 

78 

143  | 

Strain  at 

Max.  Stress 
Std,  Dev.  % 

4.1 

■ 

8.3 

. 

B 

33-f 

B 

14.5 

31.2 

B 

B 

Total 

Variation 
Impulse  ~% 

1 

0.4B 

Kfl8 

I 

■ 

1.03 

N*10 

■ 

| 

0.96 

NvlO 

1 

Propellant  D 


-25°F 

- -  - -  -  '  " 

78°F 

125°F 

Strain  at 

8.35 

5.30 

Maximum 

1 

27.50 

9.70 

Stress 

21.92 

8.27 

Standard 

14.93 

30.66 

12.58 

Deviation 

15.85 

31.41 

9.46 

AVERAGE 

16.85 

25.6 

9.1 

Total  Impulse 
Variation 

0.70  (N-26) 

This  data  is  considered  tentative;  however,  the  wide  variation  in 
standard  deviation  for  propellants  B,  C,  and  D  Is  considered  slgnigicant, 
and  attempts  will  be  made  to  explain  this  phenomenon.  Certainly  with  these 
wide  within  batch  variations  quality  control  at  the  mix  site  does  not  appear 
to  be  the  answer.  Perhaps  polymer  control  and/or  better  dispension  Of  the 
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liquid  solid  phase  and  subsequent  polymerization  Is  In  order.  It  may  be 
significant  that  the  propellant  designated  A  is  a  solution  process  end  the 
variation  of  total  impulse  might  be  even  lower  if  the  formulation  were  out 
of  the  research  stage  bb  'in*  propellant  D. 

Statistically  designed  experiments  on  a  rather  large  scale  have  been 
proposed  for  this  physical  test  study.  When  these  experiments  involve 
stamped  out  dog  bones  the  analysis  should  shew  whether  the  new  techniques 
can  bs  used  to  describe  the  variable  in  flight  testa  for  the  several  types 
.  of  solid  propellants  and  for  environmental  effects. 


PROBLEMS  IN  THE  ANALYSIS  AND  INTERWBEfRTIOM 
OF  INFORMATION  PROCESSING  EXPERIMENTS* 


Emil  H.  Jebe  and  William  A.  Brown** 
Operation*  Research  Department 
Institute  of  Science  and  Technology 
The  University  of  Michigan 


Studies  in  information  processing  are  being  conducted  by  the  Infor¬ 
mation  Processing  TaaktlPT)  of  Project  MICHIGAN  in  the  Institute  of 
Sclenae  and  Technology  of  The  University  of  Michigan.  The  experimente 
utilize  "real  world"  elements  and  simulated  elements. 

The  "real  world"  part  consists  of  a  processing  station  manned  and 
operated  by  a  six  member  crew.  On  the  other  hand,  the  non-real  parts 
comprise  a  simulated  tactical  situation  and  simulated  military  sensors 
whos^  purpose  Is  to  furnish  information  on  enemy  task  force  movements. 
A  crew  is  assigned  the  task  of  keeping  up  with  the  locations  of  enemy 
elements  in  the  simulated  tactical  situation.  Differences  between  the 
"postulated”  locations  and  the  true  positions  in  a  specific  problem  or 
run  provide  a  score  which  Is  the  measurement  of  the  performance  of  a 
crew.  This  score  is  the  response  variable  which  Is  being  analyzed. 

This  general  description  of  what  is  being  done  needs  to  be  expanded 
In  terms  of; 

1.  The  tactical  sltutation  being  studied, 

2.  The  operation  of  the  processing  station, 

3.  The  factors  which  have  been  studied, 

4.  The  Inputs  to  the  station, 

5.  The  outputs  of  the  station. 


"This  work  was  conducted  by  the  Information  Processing  Task  (IPT)  of 
Project  MICHIGAN  under  Department  of  the  Army  Contract  DA-36-039 
SC -7 8801,  administered  by  the  U.  S.  Amy  Signal  Corps. 

**Mr.  Brown  is  now  with  the  Physics  Department. 
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Understanding  of  these  live  items  will  be  aided  by  a  flow  chart  diagram  of 
the  operations,  see  Figure  L. 

The  simulated  tactical  maneuvers  take  place  within  a  military  reserva¬ 
tion  in  the  western  United  States.  The  area  oi  Interest  Is  roughly  a  square 
measuring  20  miles  on  a  side.  It  is  assumed  that  It  is  daylight  with  clear 
weather  and  the  Slue  forces  have  air  superiority.  Red  forges  consist  of 
parts  of  two  divisions  which  hava  moved  into  the  reservation  os  an  attack¬ 
ing  force.  Blue  forces  preper.e  to  repel  the  attacking  Reds.  It  is  expected 
that  Red  Forces  will  be  dispatehsd  to  counter  tha  Blue  movements.  These  • 
Red  Forces  may  move  about  20-25  miles  usually  on  roads  through  the  re¬ 
servation  area  between  1300  and  1700  hours.  A  normal  amount  of  miscella¬ 
neous  traffic  in  the  area  (called  tactical  noise)  takes  place  due  to  Red 
movements  not  directly  associated  with  the  moving  task  forces. 

It  is  the  function  of  the  surveillance  information  processing  station 
to  receive  and  process  reports  on  the  movements  of  these  Red  Task  Forces, 
l.e.,  to  track  these  concentrations  and  report  where  they  are  at  selected 
times.  "Where  they  are",  of  course,  means  where  they  are  estimated  to 
be.  Thus,  the  purpose  of  these  experiments  is  to  investigate  the  perform¬ 
ance  of  combat  surveillance  processing  system  concepts  as  Implemented 
in  a  laboratory  station.  Specifically,  measurements  are  made  of  the  ability 
of  station  personnel  to  locate  military  concentrations  as  a  function  of  sel¬ 
ected  sensor  characteristics,  the  given  tactical  situation  and  the  modes 
of  operation  of  the  station  Itself. 

In  a  number  of  experiments  five  crews  have  been  used  to  operate  the 
station  with  each  crew  repeating  the  same  problem  five  times.  The  latter 
effect  is  referred  to  as  "Repetitions."  Other  factors  that  have  been  varied 
are  the  scan  rates  for  the  two  sensors  used  and  the  detection  probabilities 
of  these  sensors.  The  number  of  moving  Red  Task  Forces  haB  usually  been 
held  fixed  for  a  single  experiment  but  has  varied  from  one  to  five  among 
experiments.  In  one  experiment  the  number  of  moving  task  forces  was 
varied  with  levels  i,  2,  3,  4,  or  5. 

An  IBM  709  computer  is  used  to  prepare  the  inputs  to  the  processing 
station.  A  set  of  computer  programs  store  the  terrain  information  and  the 
Red  Task  Force  movements  in  the  target  area.  In  addition,  the  computer 
programs  simulate  the  output  of  the  sensors  and  messages  and  overlays 
are  prepared  for  the  surveillance  station  to  process.  The  content  and  na¬ 
ture  of  these  Inputs  to  the  station  are  modified  from  the  actual  task  force 
movements  by: 
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1.  line  of  sight  considerations  which  art  checked  by  the 
computer  program, 

2.  The  particular  treatment  combinations  applied  for  the  run  . 
(experimental  unit),  which  introduce  stochastic  elements 
through  the  detection  probabilities  of  the  sensors  simulated. 

Tar  a  single  experimental  unit  (a  run  of  the  station  for  one  afternoon) 
the  information  received  by  the  station  is  based  upon  these  computer  simu¬ 
lations  and  this  information  is  used  by  the  station  personnel  to  track  the 
Red  Task  Forges. 

The  output  of  the  processing  station  consists  of  reports  on  the  location 
of  a  Red  Task  Force.  The  chief  of  the  station  crew  l «  designated  as  the 
Postulator.  He  Is  expected  to  give  the  locations  cf  the  task  forces  when 
requested  to  do  so  by  the  control  section.  These  locations  are  specified 
in  terms  of  map  coordinates  that  delineate  the  perimeters  of  the  terrain 
occupied  by  a  task  force.  Such  locations  may  be  either  an  area  or  a  route 
or  a  combination  of  areas  and  routes.  Tha  response  for  any  one  task  force 
is  limited  to  eight  map  points  for  each  prediction.  A  scoring  program  using 
Monte  Carlo  techniques  converts  the  station  outputs  Into  error  distance 
scores.  " 

In  designing  the  first  experiments,  the  Latln-Square  configuration 
seemed  moat  adaptable  to  the  Investigation  of  effects  of  interest.  Per¬ 
formance  or  employment  variations  of  the  sensors  have  provided  the  treat-  . 
ment  levels  for  tha  experiments.  Since  it  was  desirable  to  investigate 
several  factors  at  a  number  of  levels  in  one  experiment,  the  5x5  orthogonal 
square  was  chosen  as  the  basic  design.  Rows  and  columns  of  a  square  have 
been  designated  as  "repetitions"  and  "crews,"  respectively.  That  Is,  five 
crews  have  been  used  with  each  crew  being  prese-  ^ed  the  samo  problem  five 
times  but  with  a  different  set  of  treatment  combinations  imposed  for  each 
time.  From  some  points  of  view  It  would  be  desirable  to  present  a  new 
tactical  situation  to  each  crew  for  each  repetition  or  run  of  a  problem. 

In  order  to  grasp  more  readily  the  layout  of  the  experiments,  Tables  1 
and  2  are  presented.  For  the  designation  of  the  treatment  levels  In  the 
cells  of  the  orthogonal  square,  the  numbers  1,  2,3,4,  and  S  are  used.  For 
example,  crew  3  on  its  first  repetition  was  presented  with  level  2  of  Factor 
X,  level  5  of  Factor  Y  and  Level  4  of  Factor  Z. 

Within  each  cell  of  the  orthogonal  square,  i.e. ,  for  a  single  run,  usually 
eight  reports  are  made  by  the  crew  on  the  location  of  task  forces.  These  re¬ 
ports  are  spaced  in  time  at  20-minute  intervals  during  the  development  of  the 
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Table  2 

RANDOMIZATION  OP  THE  DESIGN  FOR  BLOCK  1* 


Crews 


1 

2 

3 

4 

5 

Repltltlons 

1 

3,3,2 

1,4,3 

2,5,4 

4,1,5 

5,2,1 

2 

2,4,5 

4,5,1 

5,1,1 

2,  1,2 

1.2.3 

3 

4,2,4 

5,3,5 

3,4,1 

1.5,2 

2,1.3 

4 

5,5,3 

3,1,4 

1,2,5 

2,3,1 

4,4.2 

5 

1.  1,1 

2,2,2 

4,3,3 

5,4,4 

3,5,5 

*For  Block  I  the  factors  varied  were  X,  Y,  and  Z  (refer  Table  1  above).  Thus, 
the  numbers  3,3,  and  2,  for  Grew  l,  Repetition  1  refers  to  levels  3,3,  and  2, 
respectively  for  X,  Y,  and  Z  as  described  In  Table  1.  The  factors  XJ,  V,  and 
W  were  fixed  at  levels  1,  5,  and  1,  respectively,  for  each  of  the  25  cells  of 
the  Block  I  experiment  (again  refer  to  Table  1). 
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simulated  tactical  situation.  Thus,  the  structure  o!  an  experiment  might 
be  doncrlbad  oo  orthogonal  square  with  split-plot  features  provided  by  the 
time  spacing  and  the  targets.  The  term  split-plot  is  used  because  of  the 
similarity  with  the  standard  split-plot  experiment  whether  the  main  plot 
structure  be  a  randomised  complete  block  or  Latin  Square.  Within  each  main 
plot  or  cell  of  the  orthogonal  square,,  eight  observations  are  obtained  (one 
for  each  time)  for  each  task  force.  But,  of  courBa,  neither  time  nor  targets 
can  be  randomized  as  required  for  a  split-plot  design. 

In  considering  uni -variate  analyses  of  variance  for  these  experiments, 
these  problems  may  be  stated: 

1.  The  rows  of  the  square,  designated  as  repetitions,  do  not 
conform  to  the  usual  pattern  for  rows  and  columns  in  e 
Latin  Square.  The  rows  within  each  column  may  be  expected 
to  have  some  unknown  dependence  or  correlation. 

Thi3  situation  could  be  remedied  if  different  tactical  situations  were  pre¬ 
sented  to  the  crews.  Admittedly  more  tactical  situations  could  be  developed 
for  the  one  reservation  being  used,  but  this  has  not  been  done  to  date.  We 
have  even  suggested  using  different  terrain  areas  for  each  repetition,  i.a, , 
one  situation  might  be  at  Ft.  Bragg,  another  at  Camp  Polk,  another  at  Camp 
McCoy,  etc.  Clearly,  this  would  remove  the  memory  element  for  the  crew 
In  remembering  what  happened  to  Task  Force  Alfa  on  the  last  run,  and  , 
thus,  reduce  the  unknown  correlations  la  each  column.  Such  an  experiment 
would  seem  to  be  somewhat  unrealistic,  however,  in  that  a  surveillance 
group  would  normally  function  within  a  limited  terrain  area  for  a  period  of 
time. 


2.  Degrees  ot  freedom  for  assessing  main  plot  treatments  are 
too  few.  Should  we  combine  four  degrees  of  freedom  error 
terms  from  successive  experiments? 

An  alternative  suggestion  is  to  try  to  increase  the  error  degrees  of 
freedom  within  a  single  experiment.  This  increase  may  be  accomplished 
by  breaking  out  the  Individual  degrees  of  freedom  for  the  quantitative  factors 
and  using  the  higher  order,  cubic  and  quartic,  effects  to  add  to  the  four  de¬ 
grees  of  freedom  for  error. 

First,  Table  3  presents  four  degrees  of  freedom  error  terms  from  success 
lve  experiments. 
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Table  3 

Error  Mean  Squares  from  Five  Experiment# 
lieted  by  Target  (T^«k  Force) 

Experiment 

Number  Mean  Square  for  Error* 

Target 


0 

1 

2 

3 

4 

5 

1 

2.97 

•  -  - 

2 

2.69 

■  % 

3 

1.5 

0.82 

7.2 

4 

3.36 

2.13 

0.83 

0.79 

5 

2.  1 

4.8 

2,7 

0.62 

1.1 

These  mean  squares  In  Table  3  ere  obtained  from  an  analysis  of  transformed 
data,  i.e.,  natural  logarithms  of  the  original  error  distance  scores  expressed 
in  meters.  Selection  of  an  appropriate  transformation  la  a  problem  In  Itself 
which  is  not  Included  tn  this  paper,  (2) 

On  the  other  hand,  combining  cubla  and  quartlc  effects  with  the  error 
sum  of  squares  has  been  carried  out  for  some  of  the  experiments  and  partial 
results  are  displayed  In  Table  4.  From  the  available  evidence  both  of  the 
approaches  suggested  appear  useful  for  increasing  the  sensitivity  of  the  ex¬ 
periments. 

3.  The  8pltt**plot  interpretation  for  time  as  a  factor  is  not  valid. 

4.  What  interpretations  can  be  made  if  an  overall  univariate  analysis 


♦Variable  analyzed  Is  the  natural  logarithm  of  tin  observed  error  score. 
Source:  Q) , 
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of  variance  Is  computed  -with  both  tine  acd  targets  as 
apparent  split-plot  factors  and  there  is  interest  in  inter¬ 
actions  with  main  plot  treatments? 

The  questions  (3)  and  {41  may  be  considered  together.  An  example  of  an 
analysis  of  variance  for  one  experiment  appears  in  Table  5,  below.  The  real 
problem  is  "What  is  the  proper  interpretation  of  that  part  of  the  analysis  in 
Table  5  below  the  four  degree  of  freedom  error  term?*  The  Model  implied  by 
the  analysis  seems  Inadequate  for  the  experimental  situation.  (3)  The  parti¬ 
cular  example  shown  in  Table  5  presents  no  problems;  all  the  observed  Inter¬ 
action  mean  squares  are  ‘small'  in  relation  to  the  residual  mean  square.  The 
situation  is  quite  different,  however,  for  other  experiments  of  the  aeries. 

5.  The  preceding  questions  raise  the  issue  of  alternative  designs. 

Hence,  what  designs  are  practicable  and  desirable  for  these 
experiments  ? 

Due  to  limitations  on  number  of  crews  smaller  squares,  e.g. ,  4x4, 
and  some  Youden  Squares  have  been  used.  Also,  some  non -orthogonal  de¬ 
signs  have  been  used  since  least  squares  analysis  is  easy  with  our  computing 
facilities.  (4)  The  latest  design  considered  is  an  incomplete  block  design 
from  the  class  of  partially  balanced  designs  with  two  associate  classes.  (5) 
Actually,  factorial  arrangements  of  the  treatment  combinations  should  be  used 
so  that  most  of  the  two-factor  interactions  could  be  measured.  The  5x5 
orthogonal  square  with  treatments  assigned  in  three  languages  tsinfacta  1 
in  125  fraction  of  the  total  design  and  does  not  permit"  assessment  of  any 
desired  interactions.  To  date  a  feasible  factorial  arrangement  has  not  been 
worked  out.  The  limitation  to  three  crews  is  severe.  An  examination  of  the 
National  Bureau  of  Standards  publication,  AMS  48,  for  some  of  the  smaller 
fractional  designs  indicates  that  four  or  eight  crews  might  be  used  to  block 
the  experiment  in  an  acceptable  manner.  (6)  This  blocking  procedure,  how¬ 
ever,  would  affect  the  assessment  of  the  "crew  effect"  since  crews  would 
be  confounded  with  any  other  extraneous  effects  which  the  blocks  are  de¬ 
signed  to  remove  in  evaluating  the  treatments.  It  is  believed  that  the  result¬ 
ant  confounding  would  be  no  greater,  perhaps,  thai  the  assignment  of  crews 
to  columns  of  the  Latin  Squares.  Some  extraneous  effects,  e.g.,  such  as  a 
particular  crew  always  working  on  the  same  day  of  the  week,  have  been 
present  In  the  already  completed  series  of  experiments.  On  the  other  hand, 
introduction  of  the  trick  of  a  pseudo-factor,  l.e.,  dividing  four  crews  into 
two  groups  of  two  crews  each  would  permit  direct  introduction  of  crews  as  a 
factor  in  an  experiment.  (7) 


Table  A 


ccumrsco  o?  kmh  strums -was  croic  aid  qcATvric  cowoaan» 
o?  y;,C7ca  jktmts  rsa  GEL&naa  ssPB^mcn* 


ftsPSZifiSBt  .  2«Cflt  fit  .St«5i  fcaSEft 

d.*.#  i  a  3 


3 


Error  tLS. 

A 

.  1.5 

o.6a 

7.2 

C  Ci  Q  Components 

.#* 

6 

2.4 

2.7b 

11. A 

Combined 

12 

2.1 

2.1 

10.0 

A 

« 

Error  M.8. 

4 

3.36 

2.13 

O.83 

C  &'  H  Components 

8** 

1.09 

2.3A 

6.78 

Combined 

12 

2.38 

2.87 

A.80 

5 

■ 

Error.  H.3. 

4 

8.1 

A. 6 

a.7 

C  i  Q  Components 

Q** 

0.4} 

3.6 

4.4 

Combined 

12 

1.0 

4.0 

3-8 

*  D.f.  •  dcjjrcoo  ©2  froodca 

**  Tho  composition  of  thooo  components  to  not  th«  sons  lor  all  three 
experiments.  As  so  example,  tn  Esiporioent  Number  3,  the  components 
ore  obtained  from: 

1.  Repetitions 

2.  Factor  X 

3.  Factor  T 
A.  Factor  t 


Source:  (1) 
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AJU1YSIS  6?  VARtMSCB  OP  SYSTEM  POTmWJTCK  BAS HO  Qt)  CnZCHUL  8CQBHL 
3Tv';wrcnt2D  to  iiAscm  locate  jura  csuu  the  bbccx®  model  for  moo  x 


£a»scsisa  a. l  SarJUfcAfla 

prjtrea^  of  yrapdop 

SLAv 

1LB. 

Total 

.  1 99 

153*39 

- - 

Creva 

k 

2B.*0 

6.10 

Hopotltkono 

k 

.2.62 

0.71 

Factor  7\ 

k 

3.51 

0.Q8 

Factor  Y 

k 

9.01 

2.25 

Factor  2 

k 

4.36 

1.10 

Error 

k 

11.09 

2*97 

Tlae 

7 

19*26 

2.73 

lot or actions 

XT 

2B 

10.59 

0.38 

TO 

20 

16.18 

0-58 

ET 

S3  ’ 

6.90 

0.25 

CT 

20 

21.17 

0.76  * 

BT 

SB 

0.B7 

0.30 

Residual 

26 

15.30 

0.55 

*  ftofur  Tablets  1  and  2. 
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"Models  of  wider  generality  are  described  in  this  paper.  Both  univariate 
and  multivariate  analyses  are  outlined. 
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In  discussing  this  aspect  of  the  IFT  (Project  MICHIGAN)  experiments 
it  Is  not  necessary  to  repeat  the  general  description  given  for  the  univariate 
analysis  point  of  view.(i)  A  brief  description  of  multivariate  analysis  may 
be  useful  in.  beginning  this  discussion.  For  example,  in  hybrid  com  breed¬ 
ing  work,  the  yield  of  com  per  acre  Is  usually  the  prime  variable  of  interest. 
In  some  investigations,  however,  it  is  desirable  to  consider  also  the  starch 
content,  the  oil  content  and  the  per  cent  protein  of  the  yield.  In  an  Indus- 
trlal  context,  one  may  conceive  of  bars  of  steel  being  made  up  with  varying 
alloy  contents  and  residual  amounts  of  impurities.  Then  e  metallurgist 
might  measure  the  tensile  strengths,  hardness,  and  electrical  conductivity 
of  samples  of  the  bars.  The  experimental. unit  in  this  steel  example  would 
appear  to  ba  a  batch  of  bars  and  the  sampling  might  be  done  so  as  to  enable 
the  study  of  variation  between  bars  and  within  bars  for  the  same  batch.  But 
the  response  variables  are  three — the  average  tensile  strength,  the  average 
hardness,  and  the  average  conductivity— for  each  batch. 

The  aim  of  multivariate  analysts  of  variance  is  to  make  a  simultaneous 
analysis  of  the  three  responses  for  each  batch  of  steel  bars.  Statistically, 
we  become  concerned  with  the  analysis  of  a  random  vector  rather  than  e 
single  random  variable,  say  yield,  as  is  usual  in  the  hybrid  corn  example 
described  above. 

The  essential  features  of  the  extension  to  the  multivariate  situation 
are  as  follows: 

We  have  the  ith  response  in  the  trl variate  case  as  a  vector 
(  *U*  Y2i*  *  The  expectation  of  this  vector  is  then  Wl°l 

a  sample  of  n  such  vectors  we  may  form  a  sample  matrix  of  sums  of  squares 
and  sums  of  cross-products  or  of  variances  and  co-variances.  Thus, 


*This  work  was  conducted  by  the  Information  Processing  Task  (IFT)  of  Protect 
MICHIGAN  under  Department  of  the  Army  Contract  DA-36-039  SC-78801,  ad¬ 
ministered  by  the  U.  S.  Army  Signal  Corps. 
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where  the  summation  over  1  ■  1.  Is  Suppressed  for  each  element  In  8. 
If  we  take  S/(n  -  1),  we  have 


*11 

*12 

*13 

s  » 

*21 

*22 

*23 

*31 

■» 

*32 

*33 

These  matrices  are,  of  course,  symmetrical  about  the  leading  diagonal  and 
In  a,  this  same  diagonal  contains  the  sample  variances  for  each  of  the  re* 
s ponses.  The  off  diagonal  elements  are  the  sample  covariances.  In  writing 
the  expectation  of  the  matrix  s,  a  capital  $  Is  used  and  the  •j.**  are 
replaced  by  the  (Ty's.  * 


The  description  ]uat  given  applies  to  simple  random  sampling  from 
a  homogenous  universe.  When  the  experimental  and  sampling  procedures 
are  more  complex,  the  sums  of  squares  and  sums  of  cross-products  may  be 
subdivided  in  the  usual  manner  by  the  analysis  of  variance. 


Some  further  statistical  features  may  be  noted.  In  the  expectation 
matrix,  SI  ,  If  Cy  •  0  for  1/  J,  then  the  elements  of  the  observed  vectors 
will  usually  be  Independent  random  variables.  On  the  other  hand.  If 
°1J  ^  0, '  for  some  1  and  J,  1/1,  then  the  vector  elements  will  be  correlated, 
In  the  extreme  case,  (T^/  <r  could  equal  41  or  -I.  If  this  ware  true, 
the  multivariate  analysis  would  not  add  to  the  Information.  All  the  essential 
facts  would  be  provided  by  a  univariate  analysis  of  variance  for  one  of  the 
elements  of  the  response  vector.  The  cases  of  interest  then  are  zero  cor¬ 
relation  or  moderate  correlation. 


From  the  information  or  signal  point  of  view,  we  may  say  that  in  the 
uivarlate  case,  each  experimental  unit  gives  us  three  signals.  These  signals 
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seldom  ars  Independent.  Multivariate  analysis  seeks  to  extract  more  infor¬ 
mation  from  the  combination  of  signals  than  might  be  obtained  frdm  considering 
any  one  of  the  signals. 

Details  about  computational  procedures  for  carrying  out  a  multivariate 
analysis  of  a  response  vector  are  emitted  from  this  brief  description.  Several 
sources  describe  the  computations  for  various  situations  and  purposes  (2.3, 

4,7  and  8).  An  Interesting  example  Is  treated  In  some  detail  by  Smith  and 
Gnanadeelken.  (S)  Other  examples  are  described  In  varying  degrees  of  detail 
in  the  references  noted. 

Now,  It  may  be  asked,  "In  what  way  may  we  apply  the  multivariate 
analysis  concepts  to  th.ese  IPT  experiments?"  First,  even  for  experiments 
1  and  2,  in  which  the  tactical  situation  displayed  only  one  moving  Task  Force, 
we  have  a  vector  of  observations  for  each  experimental  unit.  To  repeat,  one 
experimental  unit  was  a  single  run  of  the  station  for  one  afternoon  with  a 
given  crew  and  a  particular  repetition.  The  eight  reports  on  Task  Force  ALFA 
at  20-minute  Intervals  form  the  vector  of  observations. 

I  have  tried  to  look  at  this  Time  aspect  of  the  experiments  In  various 
ways.  As  described  In  (1)  the  structure  of  the  experiment  Is  an  orthogonal 
square  with  an  apparent  split-plot  feature  provided  by  these  observations 
spaced  in  Time  within  each  ceil  of  the  square.  Since  Time  is  not,  end  cannot 
be  in  any  sense,  randomized  as  a  factor  or  treatment  within  the  cells  the 
split-plot  approach  is  not  valid  even  though  ell  calculations  are  carried  out 
es  for  a  split -plot  experiment. 

One  type  of  multiple  response  view  of  these  experiments  Is  to  consider 
an  analogy  with  certain  agronomic  experiments.  (6)  Examples  ere  perennial 
crops  such  as  alfalfa  and  asparagus  with  several  cuttings  each  season  and 
harvest  over  several  seasons  before  a  field  is  replanted.  In  such  experiments, 
all  the  yields  over  time  may  be  added  together  for  each  experimental  unit  and 
these  unit  totals  analyzed.  Interest  in  these  experiments  also  centers  on  the 
distribution  of  yields  over  time  Oust  as  there  is  interest  in  the  fluctuation  of 
the  error  distance  scores  over  time).  Therefore,  complete  partition  of  the 
total  variation  among  the  individual  yields  is  undertaken  to  understand  the 
experiments.  Interpretation  is,  however,  complicated  by  the  correlations  tn 
time  of  the  observed  yields.  The  same  problem  exists  in  the  Information 
processing  experiments.  The  eight  reports  over  time  for  the  same  crew  and 
repetition  are  obviously  related  in  some  unknown  manner.  Adequate  replica¬ 
tion  solves  part  of  the  problem  In  some  agronomic  experiments  but  It  appear* 
that  multivariate  analysis  techniques  may  be  helpful. 
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Beginning  with  the  third  experiment  the  information  processing  ex¬ 
periments  exhibit  an  added  feature.  Multiple  targets  were  introduced,  (i.e. 
the  craws  worn  asked  to  make  reports  on  the  locations  of  three  or  more  Task 
Forces.)  Thus,  even  for  a  single  time,  say  1500,  a  vector  of  responses  Is 
obtained.  For  these  experiments  it  appears  that  multivariate  analysis  might 
be  applied  in  two  ways: 

1.  By  summing  or  averaging  over  targets  and  using  the  time  space 
as  the  vector  of  responses,  or  alternatively, 

2.  Averaging  over  time  and  using  tha  error  scores  for  the  several 
targets  as  the  vector  of  responses. 

In  full  generality,  it  appears  that  each  experimental  unit  for  experiments 
3,4  and  5  provides  a  matrix  of  responses.  This  matrix  which  is  R  by  C 
has  one  row  for  euoh  target  and  one  column  for  each  time  at  which'  reports 
ere  given  on  meat  positions.  To  dote  1  tin  not  aware  of  any  existing  meth¬ 
ods  for  dealing  with  a  matrix  of  responses  for  each  experimental  unit.  It 
has  been  pointed  out  that  the  data  may  be  vlowed  as  a  vector  of  RC  dimen¬ 
sions  for  each  experimental  unit. 

It  is  clear  from  tha  description  given  that  several  spproaohas  may  be 
used  for  analysing  tho  data  from  tha  information  processing  sxporlments 
even  though  no  methods  are  avntloblo  currently  for  dealing  with  tha  matrix 
of  responses,  Restated  these  approaches  are: 

1.  Univariate  analyses  of  variance 

e.  A  separate  analysis  for  each  element  of  the  matrix  of 
responses.  A  total  of  RO  analyses  would  be  obtained. 

b.  Analysis  by  summing  the  columns  of  the  matrix  and  consid¬ 
ering  Time  as  a  factor  in  the  analysis, 

c.  Analysis  by  summing  the  rows  of  tha  matrix  end  considering 
Targets  as  a  factor  in  tha  analysis, 

d.  A  combination  of  (b)  and  (c)  just  mentioned  with  both  Time 
and  Targets  considered  as  factors  in  tho  analysis. 
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2-  Multivariate  analyses  cJ  variance 

a.  Separate  analysis  for  each  row  of  the  matrix  {!•*•*  for 
each  target)  using  the  data  from  the  Time  space  ea  the 
vector  of  responses, 

b.  Separate  analysis  for  each  column  of  the  matrix  (l.e., 
for  each  time)  using  the  data  from  the  multiple  targets 
as  the  vector  of  responses, 

c.  Two  analyses  based  on  1.  (b)  and  Me),  above,  where 
the  response  vectors  are  In  the  Time  space  and  lntfca 
Target  space,  respectively. 


Now,  it  will  be  useful  to  consider  some  aspects  of  the  computations  la 
makings  multivariate  analysis  for  one  of  the  experiment!,  say  experiment 
four  with  four  targets  and  eight  times.  Among  the  inferences  cited,  (3)  wae 
found  to  be  the  most  helpful  in  describing  the  procedures.  Specifically,  Chapter 
7,  Section  7d  gives  the  details  for  the  multivariate  analysis  of  dispersion. 

The  distribution  theory  for  the  test  criterion  (likelihood  Ratio)  is  complex  but 
Chi  Squaro  and  Variance  Ratio  approximations  are  available.  The  appropriate 
sample  statistic  is  V  -  -m  log  \  where  X  1*  the  ratio  of  two  determinants. 

The  statlatic  V  has  an  approximate  Chi  Square  distribution  with  pq  degrees 
of  freedom.  For  pwe  may  take  thovelue  eight  or  four  depending  on  whether  we 
choose  the  Time  space  or  Target  space  vector.  For  q  we  have  the  value  4,  the 
number  of  the  degrees  of  freedom  for  the  main  effect  to  be  tested.  Thus, 
pq  «  8(4)  or  4(4).  It  would  se.em  that  a  Chi  Square  with  either  32  or  16  degrees 
of  freedom  would  provide  a  fairly  sensitive  test.  There  is  a  catch,  however. 

In  the  formula  given  for  V  there  appears  the  factor  ra.  This  ra,.n  p+a+1 

where  n  is  the  sum  of  the  degrees  of  freedom  for  treatments  * 

plus  error.  In  experiment  four,  the  n  value  4s  8  -  4  +  4,  so 


m  -  8  - 


8  4-4  +  1 
2 


•  1.5  or 


m  -  8  -  -11  4  +  1  -3.5. 

2 


Thus,  sensitivity  of  the  multivariate  test  is  measured  not  only  by  pq ,  the 
degrees  of  freedom  for  V,  but  also  by  the  vector  m  which  hee  implicitly 
embedded  in  it  the  usual  degrees  of  freedom  for  error.  Since  0  <  1,  . 
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we  see  that  a  larger  m  value  helps  to  obtain  a  significant  Chi  Square 
value. 

Alternatively,  we  might  use  tha  Variance  Ratio  approximation  Instead 
of  the  statistic  V.  The  F  obtained  has  degrees  of  freedom  pq  and  ms 
4  2  X  where  pq  Is  as  already  given.  For  ms  4  2  A  one  obtains  about  -1.95 
for  the  Tima  roaponso  vector  and  about  47,21  for  the  Target  response  vector. 
It  Is  to  be  noted  that  ms  4  2X  need  not  be  Integral  for  defining  the  degreei 
of  freedom  of  the  variance  ratio.  Since  F  is  not  defined  for  negative  de¬ 
grees  of  freedom  tha  Time  response  vector  cannot  ba  considered.  Tha  Tar¬ 
get  response  vector  might  be  considered  for  an  F(16,7.21). 

In  summary,  a  multivariate  analysis  of  variance  may  be  computed  for 
the  information  processing  experiments  using  either  the  Time  or  Target  ra^ 
Bponse  vectors.  The  V  criterion  Is  a  little  more  direct  to  obtain  in  that 
slightly  loss  computing  is  required.  For  both  statistics,  F  or  V,  the  main 
problem  is  again  one  of  Inadequate  degrees  of  freedom  for  error.  Either  the 
orthogonal  squares  used  should  be  replicated  ora  more  sensitive  design 
should  be  adopted.* 


*The  problem  of  other  designs  is  discussed  further  in  (1). 
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COMPUTATION  OF  EXPECTED  RESOLUTION  IMPROVEMENT 
FACTOR  OF  AN  INVERSE  FILTER  SYSTEM 
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Recent  research,  auch  aa  that  of  Bracewell  in  radio  astronomy,  and 
Marochal  in  photography,  has  demonatrated  the  principal  of  improving  the 
time  or  position  resolution  of  a  detection  system,  by  suitable  processing 
of  the  detector  output.  The  U.  S'.  Army  Engineer  Research  and  Developm¬ 
ent  Laboratories,  Fort  Belvoir.  Virginia,  ia  attempting  with  the  assist¬ 
ance  of  Drexel  Institute  of  Technology  to  predict  the  expected  advantages 
and  requirements  of  the  resolution  improvement  principle  under  specific 
conditions,  such  as  in  land  mine  detection  systems.  These  predictions 
are  needed  for  guidance  of  experimental  research  or.  in  the  case  of  a 
negative  result,  for  saving  the  coBt  of  an  experimental  program. 
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The  objectives  of  the  resolution  improvement  filter  are  shown  quali¬ 
tatively  in  the  first  slide.  (Slides  are  placed  at  the  end  of  this  article.) 

The  true  intensity  distribution  of  the  detected  property  is  represented  by 
,he  left  hand  view,  and  the  output  of  a  detector  of  poor  resolqtion  is  shown 
by  the  center  view.  By  passing  the  detector  output  through  a  filter  whose 
transmission  spectrum  is  the  reciprocal  of  that  of  the  detector,  onecanexpect 
an  improvement  in  resolution,  as  Indicated  by  the  right  hand  view. 
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Of  course,  we  expect  to  pay  for  this  Improvement  by  a  reduction  in 
slgnal-to-noise  ratio  and  a  loss  of  responso  accuracy.  Our  first  computation 
objective  is  to  obtain  curves  of  resolution  improvement  factor  versus  noise 
cost.  Slide  2  gives  qualitative  definitions  of  some  of  the  terms  we  use  In 
the  one  dimensional  analysis.  For  example,  the  detector  signal  from  scan¬ 
ning  an  infinitesimal  particle  is  given  In  (5),  and  the  corresponding  narrower 
filter  output  pulse  is  shown  at  (9).  The  ratio  of  these  is  the  resolution 
improvement  factor,  (1),  of  the  filter,  shown  under  (17),  at  the  lower  left 
hand  corner.  The  right  hand  column  lists  the  spectral  form  of  each  space 
function.  For  example,  the  upper  frequency  iim't  of  the  system  is  desig- 
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noted  as 


in  (10). 
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Using  these  concepts,  (Slide  3)  we  obtained  a  curve  of  resolution 
improvement  factor  1  versus  noise  cost  for  a  one  dimensional  system  by 
obtaining  each  of  these  variables  as  a  function  of  the  frequency  limit  Kg. 
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We  did  this  by  first  obtaining  the  particle  response  function  of  the 
detector  in  the  space  domain.  The  curve  Is  shown  in  Slide  4  on  a  semi- 
logarithmic  chart. 

Then  we  obtained  the  Fourier  transform  over  a  ltmltod'  frequency 
range.  The  negative  portion  of  the  curve  was  inverted  to  permit  display 
on  a  semilogerithmic  chart.  (Slide  5)  The  next, step  was  to  obtain  the 
filter  spectrum  which  Is  theoretically  the  reciprocal  of  this  curve.  Such 
a  filter  would  have  infinite  gain  at  these  zero  crossings;  therefore,  to 
obtain  a  finite  solution,  we  omitted  an  arbitrary  region  around  these  In¬ 
finite  poles. 

Then,  by  the  formulas  given  in  the  third  slide,  we  calculated  on  . 
an  IBM  650  Computer  the  resolution  improvement  factor  versus  noise  cost. 
(Slide  6)  Since  this  result  applies  only  to  a  hypothetical  one  dimensional 
system,  it  has  very  little  value  for  guidance  of  experimental  research. 
However,  the  mathematical  steps  employed  may  give  soma  Insight  into 
th 3  requirements  for  cbtdnlng  the  corresponding  expected  performance  of 
a  real  two  dimensional  system. 

Comparable  calculations  worts  attempted  for  the  two  dimensional 
filter,  and  the  spectral  values  were  obtained  for  121  points.  (Slide  7)  : 
However,  it  was  noted  that,  because  of  the  high  magnification  of  errors 
in  taking  the  reciprocals  of  low  spectral  values,  the  Integrated  noise 
results  were  extremely  dependent  upon  arbitrary  choices,  such  as  the 
relation  of  chosen  values  of  the  Independent  variables  to  the  poles,  and 
the  width  of  the  excluded  polar  regions.  For  this  reason,  these  results 
are  considered  unreliable,  and  will  serve  only  as  springboard  for  further 
research,  and  as  an  interim  guide  pending  more  accurate  results.  It  is 
interesting  to  note  that  the  area  resolution  Improvement  factor  seems  to 
follow  the  square  of  the  one  dimensional  resolution  improvement  factor. 
This  is  approximately  what  one  would  expect. 

Drexel  Institute  has  been  searching  unsucc  *  isf  ly  for  computa¬ 
tional  short  cuts  on  this  problem,  and  h"*3  Juat  recently  turned  its  attention 
to  developing  a  convoluatlon  Integral  procedure  which  would  eliminate 
the  need  for  fburier  transform  calculations.  This  study  is  still  In  progress. 

Because  of  our  failure,  after  a  year  of  trying,  to  obtain  adequate 
filter  performance  predictions,  it  seems  wise  to  proceed  with  preliminary 
experimental  research  without  benefit  of  the  hoped-for  theoretical  guidance 
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A  search  for  techniques  for  carrying  out  the  two  dimensional  inverse 
filter  process  physically  has  yielded  only  two  proposals.  One  ts  the 
optical  analog  system  shown,  in  which  the  input  is  used  to  modulate  a 
lioht  beam.  (Slide  8)  This  light  is  analyzed,  by  means  of  lenses,  into 

distribution  o(  th.  Input,  .t  wMch  point  .  sp.etnl  filter  p«- 
forma  the  necessary  filter  function  by  its  light  transmission  properties. 
Another  lens  reconstitutes  the  image  back  into  the  space  domain.  One 
limitation  of  this  system  is  the  need  for  maintaining  coherency  of  the  light 
throughout  the  process,  and  the  consequent  requirement  for  maintaining 
optical  dimensional  tolerances  on  the  light  transmission  components. 

This  limitation  is  avoided  in  the  second  proposal,  in  which  the 
light  modulation  is  maintained  in  the  spaoo  domain  throughput,  and  the 
Input  is  cross  correlated  with  the  filter  function,  also  distributed  in  the 
space  domain.  However,  in  both  systems  the  filter  function  is  bl -polar. 
We  haven't  yet  found  a  practical  way  to  accommodate  the  negative  reglona 
without  sacrificing  accuracy. 

We  will  appreciate  any  suggestions  which  may  help  us  to  complete 
the  two  dimensional  performance  prediction  computations,  or  which  may 
lead  us  to  the  best  physical  design  of  filter.  References  to  other  groups 
active  in  this  area  would  be  especially  valuable. 
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A  computation  of  one  and  two-dimensional  He  solution 
Improvement  Factor  (I)  versus  Noise. Cost*  Is  required. 

For  one  dimensional  processing. 

I »  K_21x/3.79i, 

c 

where  K  -filter  cut-off  frequency,  and 

C  « 

S  X*  width  of  detector  response  function. 
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where  Kcl  •  value  of  K  for  i  1,  and 

F(K)  ■  spectral  (Fourier)  response  of  Inverse  Biter. 

For  two-dimensional  processing,  similar  formulae  can  be 
developed  by  replacing  K  with  the  wave  number  variables  u  and 
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REFLECTING  OPTICS 

Donald  W.  Hmi 

U.  S.  Army  Ordnance  Tank  •‘Automotive  Command 
1501  Beard 

Detroit  9,  Michigan 


The  increased  need  for  be  111a tic  and  radiological  protection  in 
Ordnance  vehicles,  together  with  the  need  for  increased  surveillance 
of  the  area  surrounding  the  vehicle,  has  dictated  the  development  of  a 
viewing  system  capable  of  covering  a  large  field  of  view  from  a  small 
aperture.  In  the  recent  pa  st  many  attempts  have  been  made  at  increased 
field  viewing.  In  the  motion  picture  industry  (in  particular)  the  pursuit 
of  wider  angle  presentations  has  led  to  the  development  of  many  com¬ 
mercial  optical  systems.  The  following  table  lists  several  of  these 
systems/  together  with  their  horizontal  coverage  and  aspect  ratio. 


Name 

Horizontal  Coverage 

Aspect  Ratio 

Cinerama 

146® 

2 . 06  to  1 

Cinemascope 

“4° 

2.55  to  1 

1  Cinema  160 

160 

2.26  to  1 

Todd  AO 

128°  , 

2.00  to  1 

Clrcarama 

360° 

5.14  tol 

The  optics  used  in  these  systems  are  of  a  refractive  nature.  In  general, 
refractive  optical  arrangements  are  more  desirable  for  use  in  imaging 
because  of  thetr  compactness  and  physical  sturdiness.  However,  they 
appear  to  be  less  desirable  than  reflective  optics  for  wide  angle  imaging 
because  of  their  Inability  to  capture  extremely  wide  angle  Imaging  without 
the  use  of  several  systems  operating  in  tandem.  Refractive  systems  also 
have  rather  low  optical  efficiencies  compared  to  those  employing  reflective 
optics.  Viewing  systems  utilizing  pure  reflecting  optics  and  reflecting 
optics  in  combination  with  refracting  optics  have  achieved  fields  of  view 
up  to  360°.  Most  of  these  optical  arrangements  have  been  developed  to 
Imitate  visual  movement  in  connection  with  various  types  of  ride  and 
flight  simulators.  The  University  of  California,  Cornell  Aeronautical 
Laboratory,  Douglas  Aircraft,  and  Curtis  Wright  are  presently  engaged 
in  the  development  of  simulators  utilizing  wide  angle  visualpreaentation. 

The  viewing  system  currently  under  development  by  U.  S.  Army 
Ordnance  Tank -Automotive  Command  utilizes  a  convex  hyperbolic  minor 
as  an  image  collector  and  a  concave  ellipsoidal  surface  as  a  viewer. 
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Figure  1  Illustrates  a  proposed  application  of  the  system  In  a  closed  pod 
vehicle. 

The  hyperbolic  Image  collector  Is  mounted  on  the  vehicle  in  such  a 
manner  as  to  give  an  unobstructed  view  of  the  surrounding  area*  The  ver¬ 
tical  image  of  the  mirror  Is  picked  up  by  a  television  camera  using  a  wide 
angle  lens  and  conveyed  to  a  closed  circuit  television  projection  system. 

The  image  Is  projected  Into  the  elliptical  screen  from  the  outer  focus  of 
the  ellipse.  The  scene  Is  then  viewed  from  the  inner  focus  of  the  ellipse. 

To  date,  a  television  link  has  not  been  integrated  Into  the  arrangement. 

A'  sixteen  millimeter  motion  picture  camera  utilizing  both  color  and  black 
and  white  illm  is  being  usad  to  determine  such  parameters  as  lens  focal 

lengths  and  optimum  shapes  for  image  collectors  and  viewers. 

* 

An  illustration  of  the  typical  Image  collector  Is  shown  In  Figure  2. 

Due  to  the  geometric  configuration  of  the  real  object  and  the  virtual  Image, 
the  center  of  focus  of  the  pickup  lens  should  be  at  the  outer  focus  of  the 
hyperbola.  Location  In  any  other  position  will  tend  to  create  distortion. 

Figures  3,  4,  and  5  illustrate  three  possible  methods  of  Image  dis¬ 
play.  Projection  directly  Into  a  diffuse  elliptical  screen  (Figure  3  )  Is  the 
simplest  of  the  three  methods. 

The  Image  projector  Is  located  on  a  lino  between  the  Inner  and  outer 
foci.  The  distance  from  the  screen  to  the  image  projector  Is  determined 
by  the  focal  length  of  the  lens.  Thus,  the  shorter  the  focal  length  of  the 
lens  used,  the  closer  the  projector  may  be  placed  to  the  elliptical  screen. 

The  screen  Is  then  viewed  from  the  proximity  of  the  inner  focus.  The  focal 
spot  1b  not  critical  in  this  case  since  the  system  is  diffuse.  The  viewer 
need  only  limit  himself  to  a  spherical  area  approximately  18  Inches  In  dia¬ 
meter  surrounding  the  inner  focus.  Despite  its  simplicity,  this  method  has 
one  serious  drawback.  When  a  scene  is  viewed  In  the  lower  area  of  the 
ellipse,  the  distance  between  the  Image  and  the  viewer's  eyes  Is  quite 
small.  This  makes  eye  focus  and  convergence  rather  difficult  and  tends  to 
cause  eye  strain. 

If  the  diffuse  elliptical  screen  is  now  replaced  by  a  specularly  re¬ 
flecting  ellipsoid,  the  foci  of  the  system  becomes  much  more  critical.  The 
image  projector  must  be  located  exactly  at  the  far  focus  of  the  screen.  The 
projection  lens  muBt  then  have  an  exact  focal  length  determined  by  the  image 
required.  The  inner  focus  of  the  ellipsoid  is,  In  this  case,  a  very  sharp  focal 
point.  Since  the  focal  spot  Is  small,  viewing  this  system  necessitates  using 
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only  one  eye  at  a  time.  Thl*  condition  seriously  restricts  this  system's 
use  as  a  viewing  device.  Any  movement  of  the  viewer's  eye  from  the  focal 
spot  would  tend  to  Introduce  extreme  distortion.  Figure  4  Illustrates  the 
optical  geometry  involved  in  the  specular  ellipsoid.  From  this  figure  It  may 
be  observed  that  the  image  plane  takas  on  a  spherical  configuration  with 
the  center  at  the  inner  focus  of  the  ellipse.  The  spherical  radius  Is  equal 
to  tha  optical  distance  from  the  image  projector  to  the  viewing  focal  spot. 

If  a  diffuse  screen  is  now  inserted  Into  tha  ellipse  as  shown  In 
Figure  5,  a  combination  of  several  of  the  characteristics  of  each  of  tha 
two  previous  systems  results.  The  diffuse  screen  is  a  spherical  section 
with  a  radius  of  curvature  equal  to  tha  distance  between  the  outer  focus  of 
the  ellipse  and  the  Intersection  point  of  the  minor  axis  and  the  elliptical 
surface. 

The  image  projector  location  in  this  case  is  deparydant  upon  the  focal 
length  of  the  Ions  as  in  the  case  of  the  diffuse  ellipse  Figure  3.  The  inner 
focus  is  again  enlarged  to  a  spherical  configuration  of  about  18  inches  In 
diameter.  The  main  advantage  of  this  viewer  over  the  diffuse  elliptical 
type  lies  in  the  position  of  the  image  plane.  As  may  be  seen  in  Figure  5, 
the  Image  plane  takes  on  a  spherical  configuration  similar  to  that  In  Figure 
4.  The  radius  of  the  spherical  image  plane  Is,  in  this  case,  equal  to  the 
length  of  the  optical  path  from  the  Inner  focus  to  the  diffuse  screen.  This 
radius  is  somewhat  smaller  than  the  radius  of  the  image  plane  in  the  purely 
reflective  system.  It  is,  however,  large  enough  to  eliminate  the  eye  focus 
and  convergence  problem  encountered  in  the  use  of  the  diffuse  ellipsoid. 

It  Is  felt  that,  of  the  three  viewing  methods  previously  mentioned,  the  method 
involving  the  use  of  a  diffuse  screen  and  specularly  reflecting  ellipsoid  Is 
more  readily  adaptable  for  use  in  the  system. 

If  the  diffuse  screen  Is  removed  and  replaced  with  a  television  moni¬ 
tor  tube  having  a  face  with  a  similar  radius  of  curvature,  a  geometric  con¬ 
figuration  of  optical  paths  equal  to  those  shown  in  Figure  5  will  result.  Tha 
monitor  tube  arrangement,  shown  in  Figure  6,  is  considerably  smaller  and 
less  cumbersome  than  any  of  the  previous  systems.  The  size  and  position 
of  the  scan  lines  however,  may  cause  some  loss  in  resolution. 

Motion  pictures  using  the  hyperbolic  pickup  were  taken  from  both  a 
atationary  tripod,  as  shown  In  Figure  7,  and  a  moving  vehicle,  as  shown  in 
Figure  8.  The  location  of  the  pickup  on  a  vehicle  creates  a  problem  which 
may  cause  driver  discomfort.  If  the  image  former  were  located  In  the  driver's 
compartment  of  the  vehicle,  as  illustiated  in  Figure  8,  the  locetion  of  tha 
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pickup  and  the  Image  former  would  be  quite  different  with  respect  to  the 
center  of  gravity  of  the  vehicle.  Any  roll,  pitch,  or  yaw  encountered  by 
the  vehicle  would  have  a  magnitude  at  the  collector  different  from  that 
at  the  image  former.  The  vehicle  operator,  sitting  in  the  imager,  will  feel 
one  magnitude  of  motion  end  see  another.  This  sensation  may  cause  motion 
sickness  In  some  extreme  cases. 

The  experimental  set  up  of  the  diffuse  ellipsoid  la  shown  In  Figure  9* 

It  wos  constructed  by  molding  glass  fiber  mat  over  a  male  elliptical  form. 

The  projector  shown  is  a  16mm  Kodak  Analyst  with  a  Weinberg  Watson  Modi¬ 
fication  which  enables  the  film  to  be  single  framed  for  closer  study. 

Several  of  the  basic  problems  involved  in  the  development  of  a  panora¬ 
mic  viewer  may  be  states  as:  (a}  the  determination  of  the  optimum  shape 
and  size  of  the  hyperbola  and  ellipse;  (b)  the  selection  of  projection  and 
collection  lenses  of  the  proper  focal  length;  (c)  the  determination  of  the 
possible  problems  caused  by  the  difference  in  position  of  driver  and  Image 
collector;  (d)  the  evaluation  of  each  of  the  three  methods  of  Image  forming 
along  with  some  of  their  modifications  In  order  to  determine  the  one  best 
suited  for  this  system. 

It  is  felt  that  most  of  the  problems  in  the  system  are  caused  by  tha 
lack  of  availability  of  adequate  hardware.  In  the  future,  the  construction 
of  a  larger  hyperbolic  pickup  and  a  specular  ellipse  are  planned.  The  pur¬ 
chase  of  a  closed  circuit  television  system  is  also  planned.  Existing 
motion  picture  equipment  will  be  used  however,  until  enough  parameters 
are  established  to  accurately  define  the  characteristics  of  the  television 
system  needed. 
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I.  INTRODUCTION-  One  of  the  most  important,  yet  difficult,  phases  of 
missile  system  evaluation  is  providing  adequate  assurance  that  the  system 
will  not  cause  serious  injury  cr  death  to  friendly  troops  as  a  result  of  mis¬ 
fire,  pra -detonation,  etc.  The  primary  reason  why  this  aspect  of  tasting 
la  so  difficult  la  because  of  the  necessity  for  demonstrating,  with  e  high 
level  of  confidence,  that  the  probability  of  scrioua  injury  to  friendly  troops 
will  be  very,  very  small;  and  all  of  this  must  usually  be  based  upon  the 
results  obtained  from  a  small  sample  and  accomplished  with  a  limited  bud¬ 
get.  It  is  the  purpose  of  this  presentation  to  discuss  some  of  the  possible 
approaches  and  some  areas  in  which  it  appears  additional  atudy  and  re¬ 
search  should  be  conducted.  The  general  theme  of  this  presentation  la  to 
obtain  the  desired  confidence  In  the  weapon  safety  end  at  the  aeme  time 
keep  the  sample  size  down  to  a  reasonable  figure. 

To  make  the  examples  concrete,  it  will  be  assumed  for  this  present¬ 
ation  that  the  safety  requirement  will  be  a  99%  confidence  that  the  proba¬ 
bility  of  injury  to  friendly  troops  will  be  less  then  .0005  (one  in  2000). 

i  * 

Before  proceeding,  four  abbreviations  will  be  introduced: 

(1)  (ECIP)— Event  which  might  cause  injury  to  friendly  personnel. 

(2)  (PECIP)— Probability  that  on  (ECIP)  will  occur. 

(3)  (ED) — Friendly  personnel  ere  actually  seriously  injured  if  an 
(ECIP)  has  occurred. 

(4)  (PED) — Probability  of  an  (ED)  on  the  condition  an  (ECU)  has 
occurred. 

Inasmuch  as  this  is  a  clinical  paper,  its  purpose  is  to  present 
problems  for  solution.  These  problems  are  listed  on  the  last  sheet  (Appen¬ 
dix  A)  and  will  be  referred  to  et  the  appropriate  time  during  the  presentation. 
Actually,  M*.  the  first  item  in  Appendix  A  is  to  urge  the  group  to  consider 
the  solutions  presented  and  think  of  a  batter  approach  to  the  overall  problem 
of  safety. 

n.  THE  USE  OF  ATTRIBUTE  TESTING.  It  can  be  shown  that,  if  a  random 
sample  of  N  rounds  has  been  selected  end  tested  and  If  et  the  completion 
of  the  test,  the  number  of  (ECIP)  is  observed  to  be  f,  one  may  be  99% 

♦Numbers  in  brackets  refer  to  questions  posed  in  Appendix  A. 
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confident  that  the  (PECIF)  for  the  entire  population  oi  rounds  will  be  less 
than  .0005.  Values  for  N  and  f  are  listed  in  Table  1. 


f 

N 

0 

3,213 

•  1 

13,280 

mm 

15,820 

B 

20.100 

mam 

23,200 

TABLE  1.  Sample  Si ge  Required  to  Assure  With  a  199%  Confidence 
That  The  (PECIP)  4.. 0005. 

It  is  evident  that  the  values  for  N,  listed  in  Table  1,  are  entirely 
unrealistic  for  moot  weapon  systems.  Two  other  criticisms  are:  (1)  Such 
a  test  will  probably  not  indicate  which  sets  of  environmental  conditions  . 
will  assure  safety  and  which  will  not;  and  (2)  if  the  system  Is  not  safe, 
attribute  testing  will  not*  as  a  rule,  indicate  why  the  system  is  not  safe. 

HI.  LABORATORY  TESTING  OF  COMPONENTS.  The  second  method  will 
be  based  upon  laboratory  testing  of  critical  components.  The  approach 
will  be: 

(1)  Isolate  the  components  of  the  system  which  could  result  in  an 
(EC1P). 

(2)  Determine  those  variables  which  can  be  used  to  verify  the 
likelihood  of  the  component  causing  and  (ECU’). 

(3)  Determine  the  sets  of  environments  under  which  the  component 
is  expected  to  gperate. 

(4)  Design  an  experiment,  conduct  the  test,  analyze  the  data,  and 
attempt  to  evaluate  safety,  giving  full  consideration  to  ths 
results  of  (1),  (2),  and  (3). 


It  is  believed  that  this  technique  offers  the  greatest  promise  of  any 
suggested  within  this  presentation.  It  la  quite  possible  that  by  using  this 
method  the  desired  probability  may  be  verified  with  adequate  confidence 
and  from  a  relatively  small  sample.  A  second  reason  why  this  technique 
Is  desirable  is  because  it  may  not  only  be  used  to  indicate  whether  ths 
•wti'pott  is  m  fe  or  rrdx,  but  Ur.  vut&l  wj'  Ukobf  show  tin *  iBjntxso.  tat  eat 

safe  In  the  event  It  is  not. 
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There  are  many  Interesting  problems  associated  with  this  procedure, 
but  due  to  limited  time  I  will  proceed  to  other  techniques  without  going 
into  further  detail.  GO 

IV.  THE  APPROACH  OF  EftrAKlTlQ  DOWN  THE  CAUSES  OP  INTUKY  TC 
yKSSflDT-Y  rF.R,?-ONNgh.  fn  Section  II  it  was  pointed  cut  that  if  a  staple 
cf  S213  weapons  are  randomly  selected  end  if  none  of  these  indicate  an 
unsafe  condition,  then  we  rrry  be  99%  confident  that  the  probability  of 
an  unsafe  condition  is  l«st  than  .  Q0’05.  If  we  find  there  le  no  reasonable 
alternative  to  attribute  tasting,  one  possible  method  for  reducing  the  sam¬ 
ple  size  is  by  breaking  the  problem  down  into  two  parts.  Tlie  first  is  to 
test  the  likelihood  of  cn  event  occurring  which  might  cause  Injury  to  friend¬ 
ly  personnel  (PSCI?)  and  then  conduct  a  second  teat  to  estimate  the  proba¬ 
bility  that  if  such  an  event  did  occur  it  would  actually  Injure  friendly 
personnel  ‘  (PSD).  If  th-a  likelihood  of  either  of  these  events  oaourring  Is 
very  small,  one  might  establish  the  desired  confidence  with  quite  a  small 
sample.  For  example,  suppose  a  sample  of  nj  systems  were  selected 
and  were  operated  normally,  then  a  sample  of  n2  systems  were  selected 
and  were  Induced  to  create  a  malfunction  which  might  cauea  injury  to 
friendly  personnel  (perhaps  the  motor  might  be  induced  to  go  "high  order"), 
and  If  no  unsefe  malfunctions  occurred  in  the  first  Instance  nor  were  any 
injuries  noted  in  the  second,  then  we  may  be  99%  confident  that  n.»  ny« 
(PEC1P)  •  (PED)jJ  5.302  * .  It  is  then  clear  that  if  «,  10,604  (PECIfl 

x  (PED)  &  .0005,  then  thla  value  for  nfhj  can  be  eatlafled  if  nj  and  nj 

are  each  equal  103.  Thue  with  a  total  sample  of  206,  It  may  be  possible 
to  achieve  as  much  as  with  a  sample  of  92,13  when  the  test  is  not  broken 
down  into  two  parte. 

It  is  felt  that  the  idea  presented  in  this  section  has  a  great  deal  of 
merit  and  should  be  explored  furthor.  Actually,  one  might  conceivably 
break  the  safety  problem  down  into  three,  four,  or  more  causes  and  redqee 
the  total  sample  size  with  each  step.  The  procedure  might  easily  break 
down,  however,  because:  (1)  The  (PED)  may  be  too  large  for  the  plan  to 
be  feasible;  and  (2)  The  cost  of  testing  for  ths  (PED)  may  be  prohibitive. 

The  only  statistical  problem  1  cm  aware  of  in  connection  with  this 
procedure  is  the  limited  supply  of  tables  of  confidence  intervals  for  the  - 
products  of  binomial  parameters.  The  table  by  Buehler  is  the  only  one  with 


•See  confidence  Interval*  for  the  pmfuact  of  Binomial  Porameters,  R.  J, 
402.,,  Val-  52,  No.  ISO,  lourrosl afT the. Amertcau-abettifliiaea 
Association. 


BDsfflftgm  «ff  Ergacbnaatai 


m 

which  .1  am  familiar,  and  It  is  rather  limited.  CD 

V.  THS  USE  OF  SEQUENTIAL  ANALYSIS.  When  the  need  for  economizing 
on  sample  size  becomes  evident,  a  great  many  people  think  immediately 
of  using  'Sequential  Ant) lysis",  in  particular,  Abraham  Wald's  "Probability 
Ratio-Test".  In  fact,  the  "Probability  Ratio  Test"  has  been  somewhat  of 
a  curse  in  the  sense  that  so  many  consider  it  a  virtual  panacea  when  sample 
size  becomes  a  problom.  However,  since  sequential  analysis  is  nearly 
always  recommended  as  a  method  for  reducing  the  sample  size  required.  It 
is.  felt  that  a  few  questions  about  this  approach  should  be  presented  to  this 
group. 

We  will  proceed  by  applying  the  methods,  found  in  chapter  S  of  Wald's 
test  "Sequential  Analysis",  using  the  following  entries: 

a  -  .01  PQ  -  .0005* 

)3  -  .01  Pj  -  .005 

The  equations  for  accepting  or  rejecting  the  system  are  as  follows: 

a  -  -1.99?  4  .001956  m 
m  ■ 

d  -  41.992  4  .001956  m 

m  ,  •  * 

The  graph  of  these  equations  la  given  by  Table  2,  and  the  O.C.  Curve 
by  Table  3.  The  following  Information  can  easily  be  obtained  at  this  point: 

(1)  If  no  follwrs  occur  arhong  the  first  1019  rounds  tested,  the  system  will 
be  accepted:  \2)  The  ASN  curve  has  not  been  included,  But  its  maximum  value 
is  approximately  2000  rounds. 

Prom  these  two  observations,  it  appears  that  a  trememdous  saving  has 
been  effected  by  introducing  a  sequential  plan.  However,  if  one  Investigates 
the  O.C.  Curve  In  Table.  3,  it  appears  that  we  are  simply  trying  to  answer  the 
wrong  questions.  If  wo  are  answering  anything  at  all  in  the  area  in  which  we 
are  concerned  we  may  be  obtaining  a  99%  confidence  that  the  (PECIP)  &  .0005 
if  the  system  is  rejected. 


*1  have  chosen  these  values  because  thero  have  been  occasions  when  they 
hour  Ibfiar  goffersf  att  .the. ^ropriatte  values  to  determine,  with  a  99%  confi- 
flancsL.  that  the  lx  J’BtOEu. 
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It  Is  at  this  point  I  wish  to  ask  a  few  questions: 

fl)  Ib  there  any  existing  method  for  computing  binomial  confitienoe 
limits  by  a  sequential  approach?  $3  1  "have  presented  this 
question  because  quite  often  we  are  required  to  obtain  certain 
confidence  limits,  and  by  the  very  nature  of  the  problem  the 
sequential  approach  is  the  appropriate  one. 

{2)  Is  it  possible  to  u*e  the  well-known- "Probability  Ratio  Test* 
to  determine  confidence  limits  in  a  sequential  manner?  [Sj 

(3)  In  the  example  just  discussed,  if  at  the  termination  of  the  test, 
the  system  is  accepted,  can  we  be  99%  confident- that  the- 
(PECIP)  <  .005?  £5j  Similarly,  if  at  the  termination  of  the 
test  the  results  indicate  the  system  should  be  rejected,  can 
we  bo  99%  confident  that  the  (PECIP)  i  .00057 

(4)  If  it  is  not  possible  to  obtain  confidence  limit*  from  the 
"Probability  Ratio  Test",  are  we  not  obtaining  something  which 
is  equally  satisfying?  [6]  That  is  to  say,  we  set  up  a  test  auch 
that  the  probability  of  accepting  the  system  Is  less  than  1%  If  the 
proportion  of  failures  in  the  entire  population  exceeds  .005,  then 
the  results  of  the  test  indicated  we  should  accept  the  system.  Is 
this  not  as  satisfying  as  a  99%  confidence  limit? 

Let  us  assume  we  are  obtaining  confidence  limits,  or  something 
equally  satisfying,  from  the  "Probability  Ratio  Test",  then  It  Is  clear  that 
If  we  wish  to  answer  the'  original  question  of  this  paper  (i.e. ,  to  establish 
with  a  99%  confidence  that  a  (PECIP)  1 .0005)  It  will  be  necessary  to 
choose  fi  -  .01  and  P.  -  .0005,  while  snd  Pp  may  be  chosen  arbitrarily 
or  based  upon  some  other  consideration.  Let  us  therefore  choose  ft*  .01 
and  P0  *  .00005.  The  O.C.  Curve  for  this  plan  is  the  broken  line  In  Table 
8,  and  it  will  be  discussed  later.  The  sequential  plan  is  given  by  Table  4, 
where  It  may  easily  be  seen  that  if  the  flrBi  10,208  rounds  testod  contain 
no  (ECtP)  the  system  will  be  accepted,  if  one  and  only  one  among  the  first 
15,360  occurs,  it  will  be  accepted,  and  If  only  two  among  the  first  20,000 
occur  it  will  be  accepted.  Obviously,  this  is  considerably  moro  than  the 
sample  size  required  hy  Table  1,  and  it  is  clear  that  any  advantage  gained  by 
going  to  a  sequential  approach  is  not  found  In  reduction  of  the  sample  six*. 

There  remains  another  question  about  the  use  of  sequentlel  testing, 
lr  Jthat  -hath and  ,ma,v  be  chosen. arbitrarily.  Actually  it  makes 
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some  sense  to  choose  PQ  and  <1  small  since  the  occurence  of  a  single 
(EC, I?),  regardless  of  how  many  rounds  have  been  previously  tested  In 
which  no  (ECIP)  occurred,  will  undoubtedly  result  in  a  thorough  Investi¬ 
gation  and  perhaps  suspension  of  production  and  use  of  the  weapon* 

Consequently,  1st  us  vary  Po  and  see  what  happens*  The  number  of 
tests  required  to  accept  the  system  for  3  values  of  P0  (assuming  no  failures 
occur  In  the  sample)  is  given  by  Table  5. 


.  p0 

N 

.00005 

10,208 

'  .000005 

9,304 

.0000005 

9.  197 

TABUS  5 .  Required  number  of  tents  to  accept  the  Bystem  when 
61-  .01;  »  .01;  P}  «*  .0005;  and  no  failures  occur* 


The  following  facts  may  be  observed  from  Table  5:  (1)  the  value 
9137  la  approximately  equal  the  value  9213  listed  In  Table  1;  and  (2) 
regardless  of  how  small  PQ  la  chosen,  the  value  for  N  will  never  become 
much  smaller  than  the  9197  listed  above. 

Now,  suppose  we  vary  the  values  of  CL  .  The  required  sample 
size  for  various  values  of  &  (assuming  no  failures  occur  In  the  sample) 
Is  given  by  Table  6. 


a 

_ " _ 

a 

N 

1541 

901 

■  •  I 

212 

Hr 

tmA ;  ■ 

.9899 

22 

■ct 

iHEHbumH 

TABLE  6.  Required  number  of  tests  to  accent  the  system  when 

B  *  .01;  Pn-  ,00005;  Pi  -  ■  0005.  If  no  failures  occur1. 

Tables  7  and  8  give  the  sequential  test  plan  and  the  O.C.  Curve  for  tha 
oblvously  absurd  case  In  which  &m  .985;  fl  -  .01;  E  -  .00005  and  P.  -  .0005 
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The  results  at  this  point  appear  to  be  somewhat  ridiculous.  It  la 
certainly  Illogical-  that  by  taking  Ct  as  close  to  .99  as  we  like,  we  can 
make  It  as  small  as  we  please  as  is  evident  from  Table  6.  Furthermore, 

Table  7  is  peculiar  In  that  testing  must  cease  with  the  901  round.  That  Is, 
if  a  failure  occurs  in  rounds  one  to  900  the  system  is  Immediately  rejected, 
but  if  the  first  901  rounds  are  good,  the  system  Is  accepted. 

The  O.C.  Curve  In  Table  8  may  give  some  clue  to  the  fallacy.  While 
both  curves  pass  through  the  point  (.0005,  .01),  it  may  be  observed  that 
the  broken  line  opcrouohas  zero  rapidly  white  the  smooth  line  approaches  - 
zero  very  slowly,  after  passing  the  point  (.0005,  .01),  £73 Thus  while 
both  plans  may  give  equal  assurance  at  p  ■  .0005,  using  Ct  *  .01  will  give 
much  better  assurance  of  rejection  at  p  «  .0012.  Nevertheless,  there  appears 
to  be  a  serious  'fallacy  in  our  reasoning,  It  seems  we  are  getting  something 
for  nothing,  and  I  would  like  the  answer  why  we  can't  choose  something  like 
Ct  *  .985  and  obtain  the  desired  assurance  with  fewer  rounda. . 
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APPENDIX  A 

PROBLEMS  SUGGESTED  Wi  THE  PRESENTATION 

1.  Do  you  hrtvo  a  batter  approach  to  the  solution  of  the  safety  problem  then 
any  suggested  in  this  presentation? 

2.  Do  you  have  any  comments  concerning  the  use  of  laboratory  testing  of 
components  in  the  determination  of  safety? 

3.  To  my  knowledge  the  article  by  R.  J.Buehler,  ^Confidence  Intervals 

for  the  Product  of  Binomial  Parameters",  p.  482,  Vol.  52,  No.  280,  Journal 
of  the  American  Statistical  Association,  is  the  only  table  of  such  confidence 
limits.  Thera  Is  considerable  need  for  the  preparation  of  additional  tables  In 
this  area.  - 

4.  Is  there  any  existing  method  for  comp’'lng  binomial  confidence  limit*  by 
a  sequential  method? 

5.  Is  It  possible  to  adept  the  well  known  "Probability  Ratio  Test"  to  deter* 
mine  confidence  limits?— To  be  more,  specific,  if  at  the  end  of  the  sequential 
test  the  product  is  accepted,  can  we  be  100  (1  -|3)  %  confident  that  the 
probability  of  a  failure  is  loss  than  ?i;  and  similarly,  in  the  event  the  product 
is  rejected,  can  we  be  100  ti  -G>  %  confident  that  the  probability  of  failure 
Is  greater  than  PQ? 

6.  If  the  answer  to  5  is  negative,  are  we  not  obtaining  something  which 
may  bo  equally  satisfying  from  the  "Probability  Ratio  Test"?  That  is  to  say, 
we  set  .ip  a  test  such  that  the  probability  of  accepting  a  system  Is  less  than 
$  if  the  proportion  of  failures  in  tho  polulation  exceeds  Pj.  The  teat  indi¬ 
cates  we  should  accept  the  system.  Is  it  not  possible  this  may  be  as  satis¬ 
fying  as  a  100  (1  ~/2)  %  confidence  limit? 

7.  Is  it  conceivable  that  one  might  be  concerned  only  with  the  values  of 
/3  and  Pj  when  using  the  probability  ratio  test?  If  so,  why  can  we  alter 
the  required  sample  size  so  drastically. by  varying  CL  and  P0? 
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ABBREVIATIONS  USED  IN  THE  PRESENTATION  . 

1.  (ECIF)  —  Event  which  might  cause  injury  to.  friendly  troop*. 

2.  (PEG1P)  —  Probability  that- an  (EOIP)  will  occur. 

3.  (ED)  — ■  Friendly  personnel  are  actually  seriously  injured  if  an  (ECHO 

has  occurred. 

4.  (PEDJ  —  Probability  of  an  (ED)  on  the  condition  an  (ECU’)  has  occurred. 


DISCUSSION  OF 

"SOME  STATISTICAL' PROBLEMS  RELATED  TO  MISSILE  SAFETT" 


The  discussants  for  this  clinical  paper  Included;  Dr,  Jerzy  Neyman, 
Dr.  William.  Sechhofer,  Dr.  F.  J.  Anscombe,  Dr,  H.  A.  David,  and  Dr,  J. 

E.  Jackson.  The  suggestions  made  by  these  five  and  others  in  the  audience 
were  excellent,  and  we  have  the  opportunity  to  Include  some  of  their  com¬ 
ments. 

To  begin  with,  there  are  several  reports  which  are  related  to  this 
subject.  Some  of  these  are  listed  below; 

(1)  Ansccmbe,  F.  J.  (1949).  "Large-sample  theory  of 
sequential  estimation".  Blometrika  36.  455-8. 

(2)  Arvscombe,  F.  J.  (1953).  " Sequential  estimation* . 

T.  Rov.  Stat.  Soc.  Ser.  B  15.  1-29, 

(3)  Armltage,  Peter.  "Numerical  Studies  in  the  Sequential 
Estimation  of  Binomial 'Parameters".  Vol.  45,  Blometrlka. 

1958,  p.l. 

(4)  DeGroot,  M.  H.  (1959),  "Unbiased  sequential  estimation 
for  binomial  populations".  Ann.  Math.  Stat;  30,  80-101. 

(5)  Ray,  W.  D.  "Sequential  Confidence  Intervals  for  the  Mean 
of  a  Normal  Population  with  Unknown  Variances".  I,  Roy 
Stat.  Soc.  Sir,  a.  jj.  i3&. 
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In  addition  to  the  five  reports  listed  above.  Dr.  \T.  J.  Ans  combe  la  ' 
currently  preps  tin's  *  paper  "Testing  to  Establish  a  High  Degree  of  Safety 
in  Re-1  lability",  which  will  be  offered  for  presentation  at  certain  statistical 
meetings  and  for  publication  in  one  of  the  statistical  journals,  In  the  near 
future.  Dr.  Ansccmbe's  report  deals  with  many  of  the  questions  which  were 
raised  in  the  clinical  paper  and  gives  some  valuable  guide  Unas  toward 
their  solution. 

The  following  comments  concerning  the  cUnical  paper  were  mad#  by 
Dr.  Anscombe: 

"(1)  The  formulas  given  by  Abraham  Wald  for  the  probability  ratio 
sequential  test  are  approximations  and  should  be  used  only  whan  the  bound¬ 
aries  are  fairly  far  apart,  i.a.  when  -  h0  and  hj  are  (say)  2  or  more.  They 
are  hopelessly  inadequate  when  hQ  ■  -  .076  end  hj  ■  .002  (as  in  Table  7). 

"  (2)  The  example  illustrated  in  Table  7  ie  nothing  but  a  fixed  sample 
plan,  having  a  sample  of  size  901  and  acceptance  number  0,  with  the  und¬ 
erstanding  that  if  a  failure  occurs  before  all  901  rounds  have  bean  fired,  the 
test  may  os  well  be  stopped  at  that  point.  The  O.C.  Curve  can  easily  be 
calculated,  and  is  vary  different  from  that  shown  in  Table  B. 

"(3)  Tests  can  and  must  be  carried  out  sequenttelly.  If  the  require¬ 
ment  of  99%  confidence  that  the  probability  of  failure  will  be  less  than 
,0005  is  taken  seriously,  then  the  acceptance  boundary  must  be  Identical 
with  (or  at  least  very  close  to)  that  given  in  Table  1.  To  obtain  a  fully 
explicit  sequential  procedure,  we  must  odd  another  boundary  for  "abandon 
tho  trial".  It  will  be  proper  to  abandon  the  trial  either  because  it  seems 
likely  that  p>  .0005,  or  because  it  seems  likely  that  to  complete  the  trial 
and  reach  the  acceptance  boundary  will  be  too  expensive. 

"(4)  The  binomial  probability  ratio  sequential  test  of  Wald  la  for 
the  purpose  of  comparing  two  simple  hypotheses,  p  *  p0  versus  p  ■  pj. 

There  are  no  two  such  simple  hypotheses  here.  The  present  problem  seems 
closer  to  what  la  generally  thought  of  as  an  estimation  problem  than  to  a 
testing  problem.  Anyway,  there  is  no  reason  to  suppose  that  Wald's  type 
of  sequential  test  is  particularly  appropriate. 

"(5)  There  is  no  magical  economy  In  sequential  procedures,  such  that 
you  get  something  for  nothing.  In  a  good  sequential  plan,  observations  con¬ 
tinue  until  enough  information  is  obtained,  and  then  -tihqy  stqp.  The  only 
jRoor.oniy  its  itte  jejunum#'  uf  nut  utawpuyr  itar  asm.  liHfiara  at  aufflcsuaaair  diecLehMS 


184  *  Design  <of  Experiment* 

result  is  obtained,  nor  continuing  unnecessarily  long.  In  the  present 
case,  the  trial  should  stop  as  soon  as  one  of  the  conditions  In  Table  1 
is  met,  or  as  soon  as  the  results  are  decisively  discouraging." 

The  following  comments  were  made  by  Dr.  J.  E.  Jackson.  Dr. 
Jackson's  comments  are  direct  answers  to  questions  4-7  in  Appendix  A. 

*4.  Since  Anscombe  was  also  on  this  panel,  he  can  give  you  more 
information,  probably,  than  anyone  else  in  the  wqrld  on  sequential  esti¬ 
mation.  However,  a  few  relevant  references  might  be:  (Dr.  Jackson 

listed  several  references  which  are  Included  in  the  list  above.) 

'  r  • 

"5.  This  is  to  some  extent  covered  In  No.  4.  The  other  question 
is  whether  or  not  confidence  limits  are  really  the  '  important  thing.  See 
No.  6. 

”6.  The  point  that  Neyman  raised  is  a  good  one.  Although  1  am' 
not  sure  1  am  qualified  to  answer  it.  His  point  was  that  you  really  should 
be  using  a  significance  test  all  along  since  you  have  essentially  a  problem 
of  deciding  whether  or  not  to  use  a  particular  missile  system.  He  feels 
that  the  important  thing' is  the  decision;  worry  about  the  confidence  limits 
later.  In  this  case  you  are  testing  the  null  hypothesis: 

Ho!Tf<.0°05 

against  the  alternative 

1T2  .0005. 

"7.  If  this  is  to  be  treated  as  e  significance  test,  what  risks 
should  be  used?  Using  pi  a  .00005,  P2  ■  .0005,  CL ’■  .01  and  .01, 

you  find  you  need  from  10, 000  to  20,000  rounds .  Since  P2  and  (3  must  be. 
kept  where  they  ere,  what  happens  when  d  is  increased?  While  it  Is  true 
that  Increasing  U  to  .985  will  decrease  the  sample  size  required  consider¬ 
ably,  for  a  value  of  Pj  -  .00005,  It  would  also  mean  that  you  would  hardly 
ever  accept  a  missile  system  since  if  you  had  only  one  malfunction  in 
20,000  rounds,  you  would  still  reject  the  system  almost  ell  of  (ha  time. 
While  this  would  guarantee  yourp-rlsk  at  a  minimum  cost,  it  won't  likely 
obtain  any  improvements  in  your  missile  systems..  However,  on  page  I, 
se-wittf  paragraph  Ironr  ther  ibottosv  you  static  thunC  llh#  occurs  nee  off  a  trttnrifr 
'UTC2T  ruseTOi“Hs  n£  'new  •mtmy  muids  ’tad  lsem.  Bred,  would  result  in  an 
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Investigation. of  the  system.  In  that  erase,  it  would  seem  that  all  you 
would  need  would  be  a  single  sample  plan  which  resulted  In  rejection  If 
a  single  defective  round  was  found  In  the  sample.  However,  using 
Molina1#  Tablss,  it  appears  that  you  would  need  a  sample  else  of  9,200 
rounds  to  get  a  probability  of  .01  that  you  would  fell  to  rejeot  a  system 
having  a  .0005  probability  of  failure*  This  checks  out  pretty  well  with 
your  results  on  page  1.  Evidently,  no  matter  how  you  work  it,  you  need 
tremendous  sample  sizes  to  guarantee  the  risks  you  wish  to  impose. 

•This  might  suggest  another  possible  approach  but  X  am  certainly  not 
qualified  to  peas  judgment  on  this  One.  This. would  Involve  a  re  -eval¬ 
uation  of  the  risks,  particularly  the  choice  of  pj,  and  I  realise  that 
the  tactlos  of  war  have  changed  a  great  deal  In  the  last  twenty  years,  but 
in  ray  experience  as  a  rifleman  in  World  War  II,  it  seemed  to  me  that  the 
human  element  was  more  to  be  feared  than  the  mechanical.  By  that  I  mean, 
it  seemed  to  uh  that  we  would  encounter  more  trouble  from  wrong  firing 
orders  on  the  part  of  the  artillery  and  wrong  identification  on  the  part  of 
the  airforce  (not  that  either  occurred  very  often)  than  from  short  rounds. 

In  other  words,  one  short  round  would  not  be  nearly  as  damaging  as  one 
misdirected  aalvo.  If  things  are  still  that  way,  maybe  Is  too  small. 
Again,  this  sort  of  decision  is  not  In  my  field  but  It  Is  a  suggestion.  It 
doesn't  appear  as  though  tha  sample  size  can  be  markedly  reduced  other¬ 
wise-. 

Dr.  Herbert  Devld  made  the  following  comment: 

-I  certainly  agree  with  your  main  conclusion  that  any  type  of 
attribute  testing  would  require  an  Inordinately  large  sample  size.  Professor 
Anscombe  commented  very  adequately  on  the  sequential  procedures  you 
discuss.  In  spite  of  Armitage's  1958  Blometrlka  paper,  1  doubt  that  sequen¬ 
tial  estimation  procedures  have  a  great  deal  to  offer  over  end  above  fixed 
sample  procedures  except  as  a  by-product  of  sequential  tests.- 
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TO  BALLISTIC  DElrtCES 

13.  J.  Xatsa-nls  ami  C.  1.  Fulton 
Frankford  Arsenal . 


AgGTTl^CT.  Tha  factorial  experiment  and  Box  technique  have  bean  applied 
to  ballistic  experiments  with  reooillees  rifles,  aircraft  seat  ejection  cata¬ 
pults  and  rockets,  high-low  guns  and  Davis  gun,  for  the  purpoea  of 
reducing  time  and  cost  of  ballistic  experimental  development,  project!. 

The  result  has  been  a  reduction  in  the  number  of  round*  fired  with  little 
or  no  reduction  in  the  validity  of  the  analysis  of  variance.  A  detailed 
diecus aton  of  application  of  the  Box  technique  to  the  factorial  data  la 
presumed.  This  application  results  in  the  determination  of  a  "zona  of 
suitable  performance'*  which  makes  use  of  interaction  effects  to  provide 
greater  flexibility  In  the  selection  of  design  parameter*. 

INTRODUCTION,  In  the  experimental  development  of  ballistic  systems 
at  Frankford  Arsenal  wo  are  faced  with  a  wide  variety  of  experimental 
problems.  For  example,  in  recant  years  we  hove  been  concerned  with 
reuoUless  weapon  systems,  aircraft  oeat  ejection  catapults  and  rockets, 
thru  atari',  high-low  guns,  and  reactionless  launchers.  Some  of  these 
systems  arc  required  *to  function  repeatedly  with  performance  variations  - 
of  the  order  0.1  percent,  others  are  one-shot  devices  which  must  function 
reliably  with  performance  over,  under  or  within  certain  prescribed  limits. 
Sample  size  varies  a  great  deal,  as  well  as  tha  typo  of  performance 
requirement.  In  development  of  items  such  as  small  arms  cartridges  we 
can  fire  thousands  of  experimental  rounds  while  a  teat  ejection  catapult, 
for  example,  limits  us  to  twenty  or  thirty  to  fifty  rounds. 

By  use  of  factorial  experimental  design  techniques  and  analysis, 
combined  with  physical. Interpretation  of  the  data  in  terms  of  response 
surfaces,  as  suggested  by  Dr.  Box*,  a  tremendous  flexibility  of 
standard  statistical  practices  is  achieved.  This  method  has  been  applied 
In  one  way  or  another  to  tho  devices  mentioned  previously.  As  examples, 
our  studies  with  the  rtactlonltss  launcher,  an  analog  computer  simulation 
of  a  thruster,  and  tho  "BOX"  of  a  seat  ejection  catapult  will  be  discussed. 
The  presentation  herein,  illustrates  in  chronological  order  a  step  by  step 
experimental  evaluation  of  the  technique.  The  experimental  evaluation 


"Box,  G.  E.  F'.}  "The  Exploration  and  Explanation  of  Response  Surfaces: 
Some  General  Considerations  and  Examples",  Biometrics  Vol  10,  No.  1, 
Mir  1954. 
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was  preceded  by  an  abstract  evaluation  which  is  not  reported. hare.  Tint, 
existing  data  from  a  seat  ejection  catapult  development  was  studied  to 
determine  in  a  preliminary  way  the  method’s  effectiveness,  the  required 
type  of  experiment,  and  some  of  the  experimental  pit-falls.  Secondly, 
we  report  a  theoretical  study  of  a  thruster  from  which  we  learned  some¬ 
thing  about  the  response  surfaces  and  methods  of  interpolation.  We’ 
finally  "wrap  up  the  story"  with  a  discussion  of  the  reactionless’ launcher. 
This  study  was  conducted  from  start  to  finish  using  the  experimental 
design  methods  we  propose.  ' 

MODIFIED  M5  CATAPULT.  The  possibility  of  applying  the  Box  technique* 
to  existing  data  for  the  modified  MS  seat  ejection  catapult  was  considered. 
Although  n  carefully  controlled  experiment  as  performed  In  the  react  Ionia  a  a 
launcher  study  (to  be  discussed  later)  is  required  to  obtain  fully  valid 
results,  a  preliminary  analysis  of  existing  data  by  the  Box  technique  was 
expected  to  give  some  indication  of  its  effectiveness.  Data  from  24 
firings  of  the  modified  M5  catapult  wore  analyzed  using  three  variable*; 
temperature  (T),  charge  (C)  and  web  (W),  each  at  two  levels  for  two 
propellant  compositions  (lot  S65S.1  and  lot  5656.1).  ‘ 

The  requirements  the  modified  M5  catapult  was  to  meet  at  that  tiirit 
were  as  follows:  The  peak  acceleration  (g)  and  the  rate  of  change  of 
acceleration  (g)  were  not  to  exceed  25  g's  and  300  g/second,  respectively, 
the  final  velocity  (v)  to  equal  or  exceed  80  fps. 

The  least  square  method  was  employed  to  fit  plane  surfaces**  to  tha 
experimental  data  for  g,  g  and  v,  yielding  the  following  equation*; 

g  -  -3Q8.3W+  0. 130+  0.097T  ♦  46.1 

g  »  -1354W  +  0.358C  ♦  1.463T  +  287,6 

v  “  152. 1W  +  0.3075C +0.0908T  + 60.08 


♦Ibid. 

**The  functions  are  not  really  plane  surfaces.  To  simplify  the  calculation* 
a  limited  range  of  the  parameter  is  chosen  so  that  the  variable*  can  be 
considered  a  linear  function  of  the  parameters  within  that  range.  Caution 
must,  therefore,  be  exercised  when  interpolating  or  extrapolating.  Foe 
example,  the  origin  (WcT=C*0)  is  not  «  valid  point  on  the**  plants. 
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where  'Web*  W,  is  in  inches,  Tump,  T,  In 
and  Charge,  C,  in  gas. 

The  equations  were  plotted  for  constant  values  of  C,  W,  and  T,  i.a. , 
the  intersections  of  the  g,  g  and  v  responses  with  the  six  planes  formed 
by  choosing  constant  values  of  C,  W,  and  T  were  graphed  (Set  Figures 
1  through  3).  The  lines  on  these  graphs  represent  the  intersection  of 
the  response  surface  with  the  constant  planes.  For  example,  Figure  3A 
depicts  the  Intersection  of  the  g,  g  and  v  response  surfaces  with  the 
plane  formed  by  taking  the  temperature  ae  7Q°F.  The  arrows  indicate 
the  direction  of  Increasing  magnitude  of  the  v  response  surface  and 
decreasing  magnitude  of  the  g  and  g  surfaces. 

The  next  step  was  to  form  the  tlx  constant  planes  into  a  box.  The 
response  surfaces  within  the  cube  were  obtained  by  Joining  the  corr- 
respondlng  curves  for  g,  g  and  v.  Photo  1  (see  end  of  this  article) 
shows  this  box.  The  thickness  of  the  response  surfaces  is  a  result  of 
round  to  round  variation  in  ballistic  performance.  This  illustration  is 
qualitative,  actual  thickness  must  be  determined  fromonalyels  of 
variance  of  the  data. 

An  operating  point  (VVQ,  C0,  Tq)  whtch  satisfies  performance  re¬ 
quirements  for  this  model  must  bo  within  the  cube  volume  defined  by  the 
three  response  surfaces.  It  is  seen  that  the  g  and  &  requirements*  ere 
not  met  by  all  points  within  this  space,  except  points  in  front  of  these 
planes  (in  the  direction  of  the  arrows).  For  oxample,  the  coordinates  of 
point  WQ  -0,150  in. ,  C0  -  121  gm  and  T0  -  85°F  give  a  web,  charge, 
and  temperature  at  which  acceleration  is  less  than  25  g's,  and  rata  of 
acceleration  change  Is  less  than  300  g's/teo  with  a  velocity  greater 
then  BO  fps.  We  see  further  that  there  is  a  volume  surrounding  this 
point  over  which  the  specifications  will  be  met.  This  volume  we  will 
call  the  zone  of  suitable  response.  It  has  limiting  values  determined  by 
the  geometry  of  the  response  surfaces. 

A  bottor  operating  point  might  be  found  by  extending  the  v,  g,  and 
g  response  surfacos  outside  the  limits  of  the  Box.  For  example,  it 
appears  that  a  new  constant  web  plana  for  webs  greater  than  W  -  0*18 


•g*25  ft/second  and  g- 300  g/eecond 
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Fleur*  1  Constant  Niponia  iurf»M 
A  -  Cbarg*  a  120  gm 
B  -  Cbarca  a  130  gm 


Fleur*  2  Constant  rasponso  surface 
A  -  Web  ■  0. 16  in. 

B  .  Web  «  0.l4  In. 
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will  increase  the  temperature  range  over  which  desired  performance  Is 
achieved. 

In  addition,  the  response  surfaces  may  he  extended  In  the  direction 
of  increasing  or  decreasing  charge  or  temperature;  thus  a  volume  space 
can  be  obta!ned  over  any  desired  range  of  webs,  charge,  and  temperatures 
(other  values,  such  as  internal  volume,  expansion  ratio,  etc.,  could  be 
used  instead  of  those  chosen  for  this  particular  model)  on  the  basis  of  a 
relatively  few  firings.  Any  extension  of  the  response  surfaces  outside 
the  cube  which  represents  experimental  values  la  only  as  valid  as  the 
assumption  th3t  the  response  surfaces  are  planes.  It  becomes  important  ' 
then  to  learn  eomething  about  the  response  surface.  In  particular  the 
hazards  involved  in  interpolation  and  extrapolation  alnould  be  studied. 

A  start  was  made  in  this  direction  with  a  theoretical  study  of  a  thruster.  • 

THRUSTER.  An  analog  computer  was  used  to  develop  theoretical  response 
surfaces  for  a  thruster  which  moves  a  500  lb  load  vertically.*  Two 
restrictions  were  Imposed: 

1.  Maximum  pressure  to  be  less  than  7000  pel 

2.  Final  velocity  to  be  greater  than  7.5  fps. 

About  60  computer  runs  ware  made  for  various  design  parameter  com¬ 
binations.  The  ballistic  design  parameters  which  were  considered  are 
Charge  (C),  propellant  web  (W),  and  chamber  volume  (Vc).  The  inter¬ 
section  lines  of  the  response  surfaces  with  the  planes  were  obtained 
graphically  from  the  results  of  the  60  simulations. 

Figure  4  illustrates  the  intersection  of  the  response  surfaces  with 
the  plane:  charge  53  3  grams,  while  Figure  5  is  the  intersection  with  the 
plane:  V„  «  1.3  in.  and  Figure  6  the  plane:  Web  “  0.11  In. 

W 

*  Details  of  computer  simulation  of  ballistic  devices  can  be  found  In 
the  following  references:  Bcritz  Report;  L.  Stuart  &  W.  A.  Dittrich 
Report;  Frankford  Arsenal  Report  No.  R-1313,  "An  Analog  Computer  Study 
of  Interior  Ballistics  Equations",  L.  Stout  &  W.  A.  Dittrich;  Frankford 
Arsenal  Report,,  "Analog  Computer  Study  of  Interior  Ballistics  of 
Propellant  Actuated  Devices",  R.  Boritz  &  S.  Narise. 
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The  three  dimensional  representation  of  the  two  response  surfaces  . 
(pressure  «  7000  psl  and  velocity  -7.5  fps)  are  shown. In  Photo  2..  Some 
warping  of  the  response  surfaces  can  be  seen.  This  illustrates  a  non¬ 
linear  response.  However,  the  nonlinearity  in  well  behaved.  No 
oscillations,  peaks  cr  humps  occur.  A  linear  interpolation  should,  there¬ 
fore,  be  adequate  If  the  box  is  small  enough.  At  most,  second  order 
terms  would  be  necessary.  Size  of  the  box  should  be  small  compered  to 
nonlinearittes  but  large  compared  to  nonuniformities.'  Preliminary 
experimental  work  in  ballistic  development  should  be  directed  toward 
determining  linearity  and  uniformity.  This  information  is  essential 
before  setting  up  the  factorial  experiment  so  that  the  differences  In 
performance  levels  will  be  significant,  and  so  that  the  complexities  of 
non-linear  interpolation  cf  the  data  can  be  avoided.  In  addition,  thtt 
information  should  give  some  idea  of  the  range  of  validity  of  extrapolations 
However,  it  Is  a  good  practice  always  to  verify  extrapolation  experi¬ 
mentally.  Proper  preliminary  work  should  eliminate  the  need  for  extra¬ 
polation. 

The  operating  volume  or  zone  of  suitable  response  is  seen  to  be  ' 
triangular  in  cross  section  opening  up  in  the  direction  of  increased 
chamber  volume  and  corresponding  increased  web.  ThuB  for  an  Increased 
chamber  volume,  the  range  of  web  and  charge  over  which  the  two  re-  . 
strictlons  would  be  met  is  greater.  Picking  a  set  of  values  for  C,  W, 
and  Vc  approximately  in  the  center  of  the  zone  of  suitable  performance 
would  thus  minimize  the  chance  of  violating  our  restrictions  because  of 
manufacturing  tolerances.  A  larger  chamber  volume  would  allow  oub- 
stantlal  reduction  of  these  tolerances.  The  actual  chamber  volume 
allowable  of  course  Is  subject  to  the  physical  size  of  the  thruster  and 
other  ballistic  considerations  such  as  ignition  and  expansion  ratio. 

RFACTION1XSS  IAUNCHER.  The  reactlonless  launcher  is  a  Davis  type 
recoilless  gun  for  ejecting  masses  from  a  ballistic  missile  during  flight. 

In  the  particular  project  to  be  discussed  here,  these  masses  were  Intended 
to  decoy  anti-missile  missiles.  The  launcher  holds  two  projectiles  as 
shown' In  Figure  7. 

The  decoys  are  of  many  sizes  and  weights  and  are  launched  at  a 
wide  range  of  velocities.  The  weight  range  considered  was  20  to  60 
pounds  and  the  velocity  ranged  from  50  to  110  feet  per  second.  The  wide 
range  of  performance  required  two  types  of  interior  ballistic  systems, 
direct  and  high-low,  as  shown  in  Figure  0.  We  had  two  types  of  pro¬ 
jectiles,  Lite  bullet  type  (full  caliber)  that  fits  directly  in  the  bore  of  the 
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As  a  result  of  this  disjoining  process,  the  stud/  was  split  into  three  . 
programs,  A,  B  end  G.'  In  program  A,  a  high-low  chamber  was  used  with 
a  bullet  type  projectile.  The  main  variables  were:  . 

Charge  weight. 

Shot  “start  static  breaking  pressure.* 

Orifico  area  of  high  pressure  chamber. 
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gun,  end  the  spigot  type  which  has  a  rod  that  fits  in  the  gun  barrel  with 
the  pay  Iced  outside  the  gun,  as  shown  in  Figure  9. 

The  entire  study  Inv^ved  a  total  of  eight  variable*:  charge  type, 
decoy  type,  charge  weight,  decoy  weight,  shot-start  breaking  pressure, 
expansion  ratio,  web,  orifice  area,  To  blindly  set  up  a  factorial  experiment 
at  two  levels  would  require  the  firing  of  ?.  ,  or  256  rounds.  Replioatlng 
three  times,  which  ,1s  reasonable  for  this  type  of  study,  would  lead  to 
firing  more  than  750  (of  the  order  1O00)  rounds.  Instead  we  Isolated 
factors  with  no  interactions,  such  as  the  type  of  chamber.  The  high-law 
chamber  was  studied  separately  from  the  direct  chamber.  We  divorced 
the  spigot  projectile  from  the  bullet  type  for  the  direct  system  but  not  for 
the  high-low  system'  since  the  high-low  performance  would  not  be  expected 
to  depend  strongly  co  the  type  of  projectile. 
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In  program  B,  a  direct  chamber  was  used  with  a  bullet  type  projectile. 
The  main  variables  were: 

Charge  weight. 

Ftopellant  web. 


,*%  v 
•v  *  • 


Shot -start  static  breaking  pressure. 

In  program  C,  a  direct  cMmber  and  spigot  projectile  were  used.  The 
main  variables  were: 

Decoy  weight. 


“Shot -start  is  a  rod  which  restrains  projectile  notion  until  chamber 
pressure  reaches  a  predetermined  level. 
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Sbot-sjBit  breaking  pressure. 

Spigot  design;  l.e. ,  expansion  ratio. 


We  fired  factorial  experiment*  et  two  level*  for  these  variable#  (eight 
rounds  for  each  program).  For  the  three  program*  (A,  B,  and  C)  which  were 
replicates  three  time*,  we  fired  a  total  of  B  x  3  x  3  or  72  rounds,  e  re¬ 
duction  by  a  factor  of  10  in  the  number  of  rounds  required. 


The  discussion  is  confined  to  the  G  program,  as  this  amply. illustrate*  V 
the  important  points  and  the  other  programs  are  similar. 

The  statistical  method  used  is  found  in  Kempthorne*.  The  data  taken  t  l 
were  peak  chamber  pressures,  peak  acceleration,  and  the  muzzle  velocities 
of  the  projectiles.  In  addition,  several  other  ballistic  parameters,  such  ■ 
as  pieuometric  efficiency  and  ballistic  efficiency,  were  examined. 


Each  result  was  treated  separately,  in  the  manner  outlined  in  Keirip thorn* <  ■ 
to  obtain  the  effects  of  each  variable  and  the  interactions  between  variables.!; 
The  significance  of  these  values  was  ascertained  by  the  use  of  the  standard  '/ 
error  and  rt*  test  at  both  the  5  and  1  percent  levels.  The  values  of  the  ■ 
variables  Investigated  in  this  program  are  shown' in  Table  I  and  teat  results  : 
obtained  are  shown  in  Table  II. 

The  results  of  the  factorial  analysis  are  presented  in  Tables  m,  IV  and 
V  showing  the  effects  and  interactions  of  the  variables  on  peak  pressure, 
peak  acceleration,  and  muzzle  velocity,  respectively. 

Each  letter  in  the  table  U  used  to  represent  the  average  effect  of  the 
corresponding  parameter.  For  example  P  ■  2490  psi  In  Table  in  represents 
the  difference  between  the  average  peak  pressure  of  all  rounds  fired  with  '?■ 
a  closed  spigot  (closed  spigot  indicates  large  expansion  ratio,  consequently  !v 
this  was  considered  the  upper  level  of  this  parameter)  and  all  rounds  fired 
with  on  open  spigot.  Two  capital  letters  written  together  (WtP  for  example)  \ 
represent  the  interactions  of  the  two  corresponding  parameters.  Using  data  !; 
from  Table  HI,  WtP  -  -995  psi,  that  is,  500-2490. 


The  interpretation  of  effects  and  interactions  is  as  follows:  The  mein 
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effect  P,  for  example,  is  the  effect  on  the  variable  (Pressure  In  Table  in. 
Acceleration  In  IV,  Velocity  In  V)  of  Increasing  expansion  ratio  (dunging 
from  closed  spigot  to  open  spigot)  averaged  over  all  possible  combinations 
of  projectile  weigh*  and  shot-start  values.  It  Is  desired  now  to  determine 
the  offect  cf  expansion  ratio  averaged  over  all  shot-start  values  but  at  the 
low  projectile  weight.  This  is  denoted  symbolically  P  -  FWt. 

In  Table  JJ1,  for  example,  P  -  FWt  *  -2130  psl  indicates  that  using  data 
for  20  pound  projectile  weight  only. and  averaging  over  all  shot-start  values 
the  peak  pressure  Is  reduced  2190  psl  fn  changing  from  large  expansion  ratio 
(closed  spigot)  to  small  exr  jnsion  ratio  (open  spigot).  For  data  from  the 
60  pound  projectile  weight  and  all  shot-start  values  (symbolically  P  +  PWt) 
we  have  -2790  psi.  The  fact  that  P  -  PWt  differs  from  P  +  PWt  indicates  an  ' 
Interaction  between  projectile  weight  end  expansion  ratio. 

The  results  in  Table  Ill  show  that  Wt  +  WtP  »  200  psi  and  Wt  -  WtP  - 
610  psl.  Therefore,  the  projectile  weight  effect  when  the  open  spigot  is 
used  is  200  psi.  When  used  with  tfie  closed  iplgot,  the  projectile  weight 
effect  Is  810  psi.  The  difference  value  of  610  psl  (2790  psl  -  2160  psl  and 
910  psi  -  200  psi)  is  the  interaction  effect  between  expansion  ratio  and 
projectile  weight. 

For  a  pictorial  representation  of  the  results,  the  variables  an  laid  out 
as  the  axis  of  a  transparent  cube.  The  corners  of  the  cube  represent  the 
eight  combinations  of  variables  fired.  The  yields  (velocity,  acceleration, 
and  peak  chamber  pressure)  are  assumed  to  vary  along  the  edges  of  the 
cube  according  to  the  predictions  of  ballistic  theory.  Thus,  the  yields  at 
the  corners  are  interpolated  to.  obtain  planes  of  constaht  response.  (Ideally 
an  analog  computer  analysis  to  calculate  the  planes  exactly  1s  desirable, 
as  was  done  for  the  thruster  previously  discussed.)  The  planes  indicated 
In  Photo  3  represent  peak  pressure:  2800  psl;  velocity:  108  fps;  and  peak 
acceleration:  360  g's.  Points  within  the  transparent  cube  above  the  red 
surface  (designated  P)  represent  variables  which  result  in  pressures  below 
2800  psi.  Similarly,  points  in  front  of  the  V  surface  (green)  are  below 
10B  fps,  and  behind  the  G  surface,  are  less  than  360  g's.  .Thus,  the 
three  surfaces  enclose  a  polygon  of  triangular  cross  section  which  is  tbs 
zone  of  suitable  response. 

Combinations  of  variables  near  the  surface  of  the  zone  may  result  In 
unsuitable  performance  as  a  result  of  round-'o-rourid  variations.  Analysis 
of  variance  from  the  results  of  the  factorial  analysis  and  interpolation  of 
the  variance  along  the  cube  et»ge„  mlimg  strae  itschrilgut  a*  lux  tafias- 
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pointing  the  yields,  allows  us  to  ascribe  a  thickness  to  the  response  surfaces 
To  illustrate  this  the  zone  of  suitable  performance  has  been  removed  from 
the  cube  in  Photo  4.  The  zone  of  suitable  performance  now  appears  as 
three  boards  nailed  together.  The  hollow  space  is  known  as  the  zone  of 
acceptable  variables. 

Performance  confidence  requirements,  reliability  requirements,  and  the 
experimental  data  determine  the  thickness  of  the  surfaces.  Only  one  way 
ot  applying  this  method  is  illustrated.  The  response  surface  of  flhite 
thickness  would  be  used  to  construct  the  zones  in  different  ways  for 
different  performance  requirements. .  Suppose  the  velocity  were  required 
to  be  108  +  5  fps  instead  of  simply  greater  than  108  fps,  still  keeping  the 
pressure  and  acceleration  requirements  as  before.  Then  the  zone  of 
suitable  response  would  be  represented  by  the  green  board  marked  V  In 
Photo  4.  The  zone  of  acceptable  variables  would  be  represented  by  a 
surface  running  along  the  board  bisecting  the  thickness.  There  is  an  • 
extremely  wide  variety  of  requirements  that  can  be  treated  with  this 
technique.  No  unusual  or  exgtic*statlstical  mathematics  is  required. 

CONCLUSIONS  AND  RECOMMENDATIONS,.  Our  general  conclusion  is 
that  the  use  of  factorial  type  experimental  design  programs  represents  a 
definite  advantage  to  the  ballistic  designer.  These  advantages  are 
measured  in  terms  of  a  larger  number  of  variables  investigated  for 
fewer  rounds  (time  and  money  economy).  In  addition,  interaction  effects 
among  the  variables  are  determined.  Adding  the  Box  technique  and  pic¬ 
torial  representation  to  the  use  of  factorial  experiments  in  ballistic  re¬ 
search  gives  the  experimenter  a  more  economical  and  vivid  picture  of  how 
the  variables  operate.  To  this  picture  may  be  added  the  variances  of  each 
response.  Thus  a  zone  of  suitable  performance  may  be  determined  in  which 
the  greatest  reliability  of  operation  is  obtained. 

It  is  recommended  that  in  the  design  of  ballistic  devices  factorial  ex¬ 
periments  be  conducted  and  combined  with  a  "Box"  representation  of  the 
results. 
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I.  DIRECT  AND  INDIRECT  BUILS-UP  METHODS.'  Consider  two  types  of  . 
vapor  dissemination  trials.  In  one  type  -  the  multiple  round  -  a  rocket 
containing  a  large  number  (about  300)  agent-container  bomblets  is  fired  at 
a  horizontal  target.  The  bomblets  are  released  at  some  point  in  the  rocket's 
trajectory  and,  upon  impact,  release  their  contained  vapor  agent.  The 
bomblet  impact  points  are  determined  later  and  the  dosage  over  the  target 
area  (and  downwind  of  it)  is  determined  by  suitable  samplers. 

In  the  second  type  of  trial  -  the  single  round  -  an  amount  of  agent 
equal  to  tha  amount  contained  In  one  bomblet  is  released  instantaneously 
from  a  point  and  the  do3age  is  determined  by  suitable  samplers  over  an 
area  around  the  point  and  downwind  of  it.  The  "build-up*  problem  is  to 
use  dosage  data  obtained  from  a  trial  of  the  second  typa  to  estimate  the 
do3age  distribution  to  be  obtained  from  a  trial  of  the  first  type  under  identi¬ 
cal  meteorological  and  terrain  conditions. 

The  major  difficulty  in. this  problem  is  that  it  is  essentially  impossible 
to  test,  with  a  high  degree  of  rigor,  any  proposed  solution.  This  is  because 
identical  meteorological  conditions  can  never  be  obtained,  even  if  all  the 
relevant  meteorological  factors  were  known.  Thus,  if  a  given  method  of 
solution  does  fail  to  give  build-up  values  reasonably  similar  to  those 
actually  obtained  on  a  multiple  round  trial,  the  failure  can  be  ascribed 
eithsr  to  the  method  or  to  the  non-identicallty  of  meteorological  conditions 
and  It  is  not  easy  to  say  which  is  at  fault.  On  tha  other  hand,  it  Is  supposed. 
In  the  conduct  of  CW  trials  generally,  that  the  relevant  meteorological  fac¬ 
tors  are  being  observed  and  that  these  do  have  a  close  determining  effect 
on  the  results  of  a  trial,  for  otherwise  such  trials  would  have  no  practical 
value,  being  impossible  to  extend  toother  situations.  Hence,  if  a  build¬ 
up  method  dees  not  give  similar  results  to  a  given  multiple  round  trial,  then 
the  method  can  be  said?  at  least  within  the  framework  of  present  knowledge, 
to  have  no  practical  value. 

Furthermore,  if  similar  results  should  be  obtained,  even  in  tha  face  of 
these  difficulties,  then  it  is  logical  to  suppose  that  such  results  are  not 
simply  due  to  chance,  but  are  due  to  an  inherent  feasibility  of  the  build-up 
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method.  Therefore  with  somejeservatlons,,  it  may  be  assumed  that  a 
method  of  solution  can  be  tested. 

With  this  as  a  background,  consider  an  ideal  situation.  There  are 
two  trials  -  a  single  round  and  a  multiple  round  trial  -  with  "identical" 
meteorological  conditions  and  with  dosage  data  for  each.  The  impact 
point  locations  of  functioning  bomblets  in  the  multiple  round  trial  are 
given.  ("Dosage"  in  this  discussion  refers  to  ground  level  total  dosage.)1  •  - 
Two ’general  methods  of  building  ~up  the  single  round  data  tcthe  multiple 
round  are  suggested:  .  ' 

(1)  Direct  build-up.  Consider  a  particular  sampler  in  the  multiple 
round  trial.  The  position  of  this  sampler  relative  to  each  bombl^t  may 
be  approximately  equated  to  the  position  of  some  sampler  relative  to  the 
point  source  of  the  single  rourid  trial.'  The  dcsegc  received  at  the  mul¬ 
tiple  round  sampler  from  each  bomblet  is  then  estimated  to  be  Identical 
to  the  dosage  received  in  the  single  round  trial  at  the  appropriate  sampler. 
Thus  in  the  following  figures  let  and  B2  be  bomblet  locations  and  5 
a  multiple  round  sampler  position  and  let  P  be  the  single  round  point 
source  and  Sj  and  S2  sampler  positions.  The  vectors  BjS  and  PSj  are 
equal  as  are  the  vectors  828  and  PS2.  The  dosage  values  observed  at  Sj 


multiple  round  trial  single  round  triad 


and  S2  are  then  estimated  to  be,,  respectively,  the  dosage  contributions . 
to  S  of  and  This  process  is  extended  to  Bj,  B2.  *•  v®n  w^er8 
n  is  the  number  of  functioning  bomblets  and  the  total  of  these  estimated 
dosage  contributions  is  then  taken  as  the  estimated  dosage  at  S. 

(2)  Indirect  build-up.  A  functional  form  is  fitted  to  the  single 
round  data  and  this  function  Is  used  to  estimate  the  dosage  at  any  point  _ 
in  the  multiple  round  trial.  For  example  suppose,  from  the  single  round 
dati.  a  fiitaotioa  D(x,y)  is  fitted  whisk  pi. *es  the  value  of  the  dosage 
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at  downwind  distance  x  and  croaswind  distance  y  from  the  point  scarce. 
Then  if  the  bomblets  on  the  multiple  round  trial  are  located  at  txj.y]), 
(xjj.yJ.  ^e  built-up  doeage  at  (x.y)  is  simply 

JS  (*.y)  -  X  D(x-x^.  y-y.)  (9 

i-l 

Further,  with  a  large  number  of  bomblets  it  is  feasible  to  assume  a 
bomblet  distribution  density  f(x,y)  where 

JfR  ftx.y)  dx  dy  -  1 

and  R  is  the  bomblet  impact  region.  Then  Instead  of  (1)  ere  may  take 

«0(x,y)  *  n//  f(u,v)  D(x-u,y-v)du  dv  (2) 

"R 

The  advantage  of  equation  (2)  over  equation  (1)  is  that  the  use  trf 
equation  (2)  will  usually  not  require  specifying  n  bomblet  coordinates. 

For  example,  if  f(u,v)  is  a  bivariate  normal  density  (perhaps  truncate# 
then  equation  (2)  is  specified  by  the  one,  two,  or  et  most  three,  para¬ 
meters  of  f(u,v).  Since  these  parameters  are  relatively  constant  for  a 
given  ballistic  situation,  equation  (2)  will  have  much  wider  predictive 
ability  than  will  equation  (1).  Specifically,  it  can  be  used  to  predict, 
prior  to  the  trial,  the  results  of  a  multiple  round  trial  provided  ballistic 
Information  on  bomblet  impact  pattern  distribution  is  obtained.  As  a  matter 
of  fact,  build-up  methods  may  turn  out  to  have,  es  their  primary  function, 
usefulness  in  guiding  the  conduct  of  multiple  round  trials,  rather  than  In 
replacing  them. 

II.  COMPARISON  OF  THE  METHODS.  There  are  several  reasons  which 
suggest  that  the  direct  method  of  build-up  or  some  modification  of  it 
will  fail  to  give  useful  results: 

(l)  Immeasurable  single -round  dosages.  An  obvious  difference  between 
single  and  multiple  round  dosage  results  is  that  the  apparent  area  of  cloud 
travel  is  considerably  smaller  for  the  single  than  it  is  for  the  multiple 
round  situation.  This  is  caused  by  the  fact  that  at  the  single  round  cloud 
edge  dosages  are  sc  sisali  as  to  be  im»ee;iwralile  by  the  analytic  pro¬ 
cedures  employed.  Such  small  individual  bomblet  contributions  become 
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measurable  and  Important  In  the  multiple  round  situation,  however.  Thus; 
in  terms  of  the  above  figures,  the  dosages  observed  at  Sj  or  S2  could  be 
zero  and  this  would  lead  to  a  definite  under-estimation  of  the  dosege  et 
S.  To  a  certain  extent,  this  disadvantage  is  shared  by  the  Indirect  method  • 
but  not  to  as  great  an  extent,  since  the  estimate  of  D(x,y)  uses  data  from 
"measurable"  areas. 

(2)  Single  round  samplers  unavailable  at  desired  locations.  Since 
the  locations  of  Bj,  B^,  .....  Bn  are  random,  it  is  necessary  to  take 
Sj,  S2,  .....  Sn  to  rpake  the  vectors  BjS,  .....  BnS  as  only  approximately 

equal,  respectively,  to  PSj,  .....  PSn.  Unless  the  single  round  sampling 
grid  is  very  dense,  these  approximations  will  be  quite  rough.  Further,  the 
area  of  primary  interest  in  the  multiple  round  situation  has  been  taken  to  be 
at  considerable  downwind  distance  from  the  Impact  pattern.  In  this  area 
the  dosage  contributions  of  individual  bomblets  is  small  and  it  is  In  this 
corresponding  area  In  the  single  round  trials  that  the  sampling  density  Is 
low.  In  particular  the  downwind  distances  between  successive  single  round 
sampling  arcs  is  large.  The  dense  sampling  array  found  close  to  the  release 
point  of  single  round  trials  does  not  contribute  much  to  the  total  accuracy 
of  build-up  for  large  downwind  distances.  Conceivably,  the  direct  method 
could  be  greatly  improved  by  having  dense  sampling  at  large  downwind  dis¬ 
tances  in  the  single  round  trials.  This  would  not  help  to  answer  reason 
(1),  unless  the  analytic  procedure  were  made  more  sensitive.  In  any  case, 
the  indirect  method  does  not  face  this  disadvantage  at  all  since  D(x,y) 
is  defined  at  every  (x,y).  ‘  -  . 

1 3)  Sampler  variability  and  cloud  heterogeneity.  The  direct  method 
develops  estimated  dosages  by  local  build-ups.  Heterogeneity  Is  a  local 
phenomenon  and  can  be  ameliorated  only  by  a  statistical  "smoothing"  pro- 
cuss.  This  means,’ in  effect,  that  a  process  such  as  the  Indirect  method, 
which  uses  all  the  data  to  estimate  each  point  Is  better  than  a  process 
which  estimates  each  point  by  an  individual  observation. 

(4)  Won-predlctahlllty  of  results.  An  advantage  of  the  Indirect  method 
is  that  equation  (2)  can  be  used  to  replace  equation  (1)  and,  as  has  been 
discussed,  lead  to  predictions  Independent  of  knowledge  of  bomblet  loca¬ 
tions.  This  1b  not  feasible  wtth  the  direct  method,  unless  some  complex 
analog  (such  as  assuming  a  bomblet  pattern)  were  used. 

In  summary,  It  can  be  said  that  the  direct  method  of  build-up  is  In¬ 
herently  Incapable  of  giving  good  estimates  of  multiple  round  dosages 
and  'lead’s  to  less  useitll  fljux  baimj  eioa-pcedlcta blue))  results. 
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HI.  APPLICATION  OF  THE  INDIRECT  METHOD.  The  basis  of  the  indirect  ■ 
method  is  the  Calder-Sutton  instantaneous  point  source  model: 

D(x,y)  -  exp  [a-by2/**6 -c  Jn  (x)]  (3) 

where 

'  D(x,y)  “  ground-level  total  dosage  at  downwind 
distance  x  and  gross  wind  distance  y  and  a, 
b,  c,  are  parameters. 

•  *  i 

Th«?  overall  procedure  of  the  indirect  method  is  as  follows. 


(1)  "Fit"  the  model  to  the  data  of  the  single  round  trial.  This 
amounts  to  an  estimation  (say  by  least  squares)  of  the. parameters 
a,  b,  c,  c(. 


(2)  Apply  equation  (1)  or  alternatively  equation  (2)  (if  a  bomblet 
density  function  f(u,v)  can  be  assumed). 

The  problem  of  fitting  the  model  is  lengthy  and  will  be  discussed 
after  the  second  problem  -  application  of  equation  (1)  or  (2)**  Is  considered. 
Therefore  to  start  this  discussion,  let  us  assume  that  the  function  D(x,y)  . 
has  been  estimated.  Note  that  equation  (3)  is  defined  only  for  x>  0. 

(This  means  that  the  model  assumes  no  upwind  dosage,  an  assumption  which 
Is  not  strictly  true,  but  which  can  be  considered  as  having  a  compensating 
error  due  to  the  fact  that  upwind  dosages  will  be  ignored  in  both  the  single 
round  and  the  multiple  round  situation.)  Hence  in  applying  equation  (1) 
we  must  take: 


&  (x,y) 


-yWy  y-yt) 


Xf£x 


(4) 


as  the  estimated  built-up  dosage  at  (x.y).  Nothing  more  can  be  said  about 
this.  We  do  not  recommend  use  of  this  procedure.  Inasmuch  as  it  is  tedious 
(although  quite  suitable  for  solution  on  a  high-speed  computer)  and  does  not 
lead  to  "predictability".  We  therefore  proceed  to  the  more  interesting  ques¬ 
tion  of  application  of  equation  (2). 
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.  A.  Normally  Distributed  Botnblet  Pattern. 

■Suppose  we  assume  that  f(u,v)  Is  a  bivariate  normal  density 
(distributed  around  the  Impact  pattern  center): 


f(u.v) 


exp< 


r 

1 

(A. 

v2 

.  1 

2Puv  1 

2(1-P2) 

b 

X 

h  T 

°V 

crx<ry  J 

&> 


.  Then  we  take,,  for  equation  (2), 
-£ 


M-a  fot> 

•0(x,y)*n\  l  *  f(u,v)  D(x-u,y-v)dv 

ec  J-  co 


(6) 


Taking  the  crosswind  limits  of  Integration  as  -«j°and  +*°  Is  a 
simplifying  approximation  and  should  not  make  much  difference  In  the 
results  since  the  tails  cf  the  normal  distribution  rapidly  become  small. 
Note  that  the  domwind  upper  limit  of  integration  Is  x-£  *  This  Is 
because  the  integrand  Is  discontinuous  at  u**x  In  that 


lim  D(x-u,y-v)  »o°;  V»y 
u-*x- 

**  0;  vffy 


The  problem  is  to  choose  a  reasonably  small  S'. 


If  we  consider  the  bomblets  upwind  of  x,  it  is  clear  that  the  Inte¬ 
gration  should  be  limited  to  the  most  downwind  of  these  bomblets.  In 
fact  It  Is  reasonable,  in  making  practical  use  of  equation  (6),  to  Integrate 
up  to  the  expected  downwind  coordinate  of  this  most  downwind  bomblet. 
This  concept  is  difficult  to  explain  briefly  without  the  following  mathe¬ 
matical  development. 

Let  x  be  any  downwind  axis  coordinate  and  let  nW  be  the  number 
of  bomblets  upwind  of  x  (l.e„ ,  the  number  of  bomblets  whose  downwind 
axis  coordinates  are  to  the  left  of  x.)  Lot  u(x)  be  the  largest  of  these 
coordinates  (i.e. ,  u(x)  is  the  largest  downwind  coordinate  of  those  bomb- 
lets  which  ure  upwind  of  x).  Now  u(x)  in  a  random  variable  and  It  seems 
reasonable  to  take  equation  (6)  as 
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„  r  U  W  -*> 

<°(x,ri,n  du  f  (u,  v)D  (x-u  ,  y-v)*r 

J  —oo  J-oo 


where  ufx)  »  Eu(x)  is  the  expected  value  of  u(x),  for  a  given  x.  Con¬ 
sider -now  the  evaluation  of  u£x). 

First,  the  marginal  probability  distribution  function  of  bomblet 

downwind  axis  coordinates  is  . 

.  , 

f*  r°°  \ 

F(x)  »  |  du  J  f(u,v)dv 

J-oo  J-oo 

and  so  the  probability  of  any  given  value  of  n(x)  is  given  by  the 
binomial  distribution  as 

- 

p(n(x))  -  (£(*))  [f(x)]n^  [l-F(x)J  n"nW.  nW  -  0,1,2,  , ,  n. 

Now  u(x)  is  defined  only  when  n(x)  >1  the  probability  of  which 

is 

1-  [l-F(x)]n. 

Hence  the  condltionel  probability  of  any  given  value  of  n(x>  under  tha 
condition  that  n(x)  ^1  is 

p.(.wi  -  <!!(,,>  [rw]"M  [i-rwl—w  m 

l-[l-F(x)J® 

n(x)  -1,  2,  .....  n. 

The  conditional  probability  distribution  function  of  u(x),  for  a  given 
value  of  n(x),  is 

G(u(x)|n(x»  ■  [~F^X^ - ]  !  “W^*  (9) 

Hence  the  marginal  probability  distribution  of  ufx)  is 
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G(uCx))  «  £  P*(nW>6  [uW  jn«i] 

n(xH 

,  1  .  irfrr  M  -rag.-  .  .  u(x]4  X.  m 

l-[l-F{x)Jn 


Therefore 

u(x)  “  Eu(x)  *  f~  udG(u).  (U> 

-'-oo  • 


Unfortunately,  this  integral  is  not  simple  and  can.be  evaluated  only  by 
tedious  numerical  methods.  It  seems  reasonable  therfare  to  a p proximate 
u(x)  by  fl(x)  where 

r[Q(x)]  «  E  Fju(x)j 

-  f  I‘(u)dG(u)  -  i  ^r(u)  [nr(u)-FW]n'1dfM 

n  +  1 

1  -fl-F(x)]n 


-  FW- 


i  -fi-r(xj|n41 
n  •+ 1 


ri-Fwin 


1  -[l-F(x)]n 


t 


rw 


.  .l-ti-Ffri' 


rr+1 


n  1 


] 


Thus  we  take  u(x)  as  the  solution  to 


0: 
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which,  since  the  integral  F(x)  is  tabled,  is  not  difficult. 

For  cases  where  F(x)>  1/2  (i.e. ,  x  is  downwind  of  the  bomblet 
downwind  axis  mean)  and  reasonably  large  n,  .equation  (12)  is  very  well 
approximated  by 


and  instead  of  (13),  take 

F[u(x)]  -F(?t)  (I3e) 


As.  previously  defined, 


F(x)  »  {  du  f  f(u,v)dv 
J-ca  •  J~oo 


which,  from  equation  (5),, 


"vPT  L’T*  «*p(-u2/2}<fc.-N  (-£-),  ur. 


.  ya  J-oo  " ' 

The  Integral  the  normal  probability  Integral,  Is  well  tabled.  Thus, 

If  equation  (13a)  is  used,  the  value  u(x)  such  that 


"It1] 


N(“~)  - 


is  obtained  for  each  x.  In  any  case,  values  of  u(x)  are  computed  for 
corresponding  values  of  x  and,  instead  of  equation  (7),  we  take 


,U(x)  r<X>  . 

*o(x,y)-n[  du  /  f(u,v)D(x-u,y-v)d7 
J-oo  J-n 


Now,  from  (3)  and  t5). 
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r< 


f  (u,  v)D(x-u,  y~v)dv 

oo 


exp{a-c  AU-u) 


2u-r»  <rJ 


} 


2^0-  V!  - 


V  r*exp  /-  »*-A— -■  feL  -  l&ttfcJ-  JitolLi) 

t  z{[  f  *  r2  CTxOV  J-  (x-u^J 


dv 


k  exp 


f- 


2(l-pV.2,  ' 


•<x-uF  2h  J 


Gt-u)c  Vh 


g{u,x,y),  say 


where 

k 

end 


wvWo^T 


2b 


(l-f2)^  <x-u)« 

so  that 

f«W 

•P  (x,y)  *  n  I  g(u,x,y)du 

90 


(16) 


(17) 


The  problem  then  reduces  to  evaluating  equation  (17)  for  a  "grid"  of 
points  (x.y).  The  Integral  la  not  simple  and  requires  tedious  numerical  methods 
which  will  not  be  explored  here.  However,  a  reasonable  approximation  can 
be  given  briefly.  The  function  g(u,x,y)  can- be  factored 

g(u,x,y)  -  gj(u,x,y)«J2(u,x,y) 


where 
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S2(u,x,y)  - 


k  exp  j 

(  bv21 

L 

,  j  .Pu  .  ,2yb.  1 

1  _ 2h _ 

(x-u)c  \ 

fiT 
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then,  by  the  mean  value  theorem,  there  exists  u*  {-oo  <  u*  .<  uW)  such 
that 


ft  (x,y) 


n&2(u*,x,y)  j  ®1  (u,x,y)du 


where,  approximately* 


u* 


/.UJ^U9l^u'*,y^ 


✓“to 

I  5»j  {u,x,y)du 

■'-'50 


Little  further  eTor  Is  introduced  by  replacing  u(x)  by  x  in  equation  (19), 
and  this  makes  the  calculation  of  u(x)  unnecessary.  Thus, 


(  g^u,  x,  y)du  - 


(20) 


where,  as  before,  N  is  the  normal  probability  Integra], 


i _ (*,tr*  VT-?,T "  -M 

JW  i-«  .  *  * 

r 


Also, 


r 

j- CO 


ugA(u,x,y)du  » 


<*-  f2,*4  -r?»v> ) 


(21) 
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If  a  circular  impact  pattern  is  accepted  a*  the  most  practical  case, 
equation  (23)  is  the  most  useful  build-up  equation  so  far  developed  in  this 
paper.  It  ts  interesting  in  that  it  may  also  be  used  as  a  model  for  multiple 
round  data.  The  values  of  x  end  y  are  respectively  the  downwind  end 
crosswind  distances  from  the  impaet  pattern  center. 


B.  Use  of  Equation  (23)  as  a  Multiple  Round  Model. 


It  Is  not  entirely  out  of  place  at  this  time  to  discuss  equation  (23)  as 
a  multiple  round  model,  having  five  parameters:  a,  b,  c.d.O".  lithe 
bomblet  positions  (xj,yi),  .....  (xn,yn)  are  measured  from  the  pattern 
center  then  a  simple  estimate  of  the  parameter  a  is 


Now  consider  certain  functions  derived  from  equation  (23).  Pint, 
for  a  given  downwind  sampling  row,  we  have  the  “crosswind  maximum 
dosage*:  • 

n«‘  N(J8.) 

CWMDM  -^(x,0)  -  - - - — ; - mass; 

(x“U*)c”°72  V(x-u*Jt*+  2bar  (26) 


and  the  “cresswind  integrated  dosage*: 


CWlD(x)  - 


(27) 


(Each  of  these  quantities  is  obviously  observable  experimentally,  for  each 
downwind  sampling  row.)  The  estimate  of  makes  N(x/V)  observable  a* 
Is  (see  equation  (24))  u*.  Then  various  functions  of  (26)  and  (27)  maybe 
plotted  against  (x-u*)  to  obtain  estimates  of  b,c,  andct.  For  example, 
consider 


CWIP(x)_)2  _  2cr? 
mvwn/vr  '  ^ 


(28) 


238 


Design  of  Expertmtt&s 


so  that  If  z(x)  is  plotted  against  x-u*  on  log  log  paper  a  straight  line 
should  result  with  intercept  *  log  (1/b)  and  slope  -  CL.  The  success 
of  such  techniques  will  be  very  much,  dependent  on  the  reliability  of 
the  dosuqe  data.  Nothing  further  will  be  said  on  this  subject  In  this 
paper,  inasmuch  »s  the  present  problem  is  not  the  estimation  of  para¬ 
meters  from  multiple  round  data,  but  is  rather  the  “building -up". of  single 
round  data. 

C.  General  Impact  Pattern  Distributions. 

Suppose  that  at  the  time  of  release  of  bomblets,  the  rocket  is  at 
position  (x0ly0,2()  and  suppose  a  given  bomblet  has  Initial  velocity 
vector  (vx.Vy.,vz).  Then  v/ith*the  effects  of  wind  ignored,  the  velodlty. 
vector  of  the  bomblet  at  time  t  after  release  is 

<VVv*  ■ 50  •'  ' 

where  g  Is  the  gravity  acceleration  constant.  The  position  of  the  bomb- 
let  at  time  t  is 

<VV*0)+f  (Wvz  *  " 

o 

(xo+vxt,yo+V'  W  "  0t^2)  (29) 

The  time  of  ground  impact  is  such  that  the  vertical  coordinate  is  zero. 

That  is,  the  impact  time  Is  such  that 

zo  +  V  "  gt 1/2  “  0 

the  positive  root  of  which  is 


9 


The  ground  position  of  the  bomblet  Is  therefore 

*l‘xo4Vl 
yl  =  yo*  Vl 


Tj  &  sUyii  ^  ^xtMlsee  rtt* 
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Conceivably,  a  bornblet  pattern  distribution  can  be  deduced  from 
simple  considerations  cf  equations  (30)  and  (31)and  of  the  distribution  of  , 
Xo.y^.x  ,vx,vy,  and  v..  For  example.  If  xjj,y0,  and  zp  are  considered  •  • 
non-random,  and  the  enact  of  v2  Is  ascii  relative  to  the  effect  of  gravity, 
and  If  vx  and  Vy  are  bivemte  normally  distributed  then  x .  and.  y^  are 
bivariate  normally  distributed.  We  are  continuing  our  exploration  of  this 
problem. 

nr.  TTTTTyC"  THE  SINGLE  PQTWPD^TA.  Consider  now  the  problem  . 
where  total  close  go  dots  hi:s  bi£n  obtained  over  a  sampling  grid,  in  a 
trial  of  ihe  second  type  -  the  single  round.  If  a  v/lnd  direction  can  be 
assumed  and  if  the  sampling  grid  is  such  that  itccntalns  rew*  perpen¬ 
dicular  to  the  v.'iiid  direction,  the  model  fitting  is  greatly  simplified. 
Consider  the  dosage  model 

D(x,yJ  *  exp(a-by2/xct  -c  in  x) 


shews  that  if,  for  a  constant  x,  the  negative  of  the  log  dosage  is  plotted 
against  y2,  the  rquare  of  the  crosswind  distance,  a  straight  line  with 
slope  ■  b/x^  should  result: 


log  D(x,y) 


If  this  slope  is  estimated  for  each  crosswind  row  and  if  the  set  of  slopes 
Is  plotted  against  x  on  log  paper,  a  straight  line  with  slope  ■ 
should  result; 


4 
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By  this  Means  an  estimate  of  dean  be  obtained. 

After  d  Is  estimated  the  estimation  of  a,  b,  and  c  Is  quite  simple 

inasmuch  a3  ordinary  least  squares  techniques  can  be  applied  to  • 

-  *** 

z{x,y)  =  B-b/-cm 
xvhere 

z{x,y)  «./n  D(x,y) 

Ay2/,5 

in  ■»  /n  x 


Thus  the  least  squares  estimates  are 
a  ■  2  +  $,/-♦■  cm 

£  ■  -  (Vz 

2 

£  "  (s,  9-  -  V/3  )/Syy*  (32) 

/z  m  //  mz  //  mm  /m 

where  z,  2,  in  are  the  means,  respectively  of  the  z's,  /'*.  and  m's 
and 

s/z-  I(/-/)(z-  z) 

*  E(/-/)(m-m)  - 
Syy  -  ZU-I)2  ,  etc. 

There  is  one  difficulty  with  application  of  the  model  to  such  data. 

The  model  u_ scribes  diffusion  due  to  wind  currents.  "Close"  to  the  release 
point  another  mechanism  -  the  munition  blast  -  becomes  more  important  in 
cloud  travel.  Therefore,  not  all  the  data  Is  suitable  for  use  in  equations 
(32).  Some  Judgment  about  this  may  possibly  be  obtained  by  observing  the 
plot  of  log  slope  against  log  x  indicated  above.  For  small  values  of  x. 
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the  plot  may  show  erratic  departures  from  linearity  and  this  is  an  Indication 
that  data  from  the  corresponding  crosswind  towb  is  unsuitable  for  fitting. 

Other  observable  functions  can  be  used  for  aiding  in  the  fit.  Far 
example  the  crosswind  maximum  dosage; 

CWMDW«D(x(0)»«a/xc 


and  the  crosswind  integrated  dosage: 


CWID(x)  » 


so  that 

CVhQM  ,  r.°yrTT,/2~ 

GWMDW  ■"  b 


(33) 


A  plot  of  this  latter  quantity  against  x  on  log  paper  at  once  gives  an  esti¬ 
mate  oi  CL 


The  above  estimation  procedures  depended  on  having  crosswind 
sampling  rows  perpendicular  to  the  wind  direction.  In  fact,  this  will 
seldom  be  the  case.  The  problem  Is  partially  avoided  by  having  circular 
sampling  arcs,  but  this  still  leaves  some  difficulty.  An  approximation 
suggests  that  a  circular  arc  with  large  radius  will  exhibit  similar  dosage 
results  10  a  straight  line  perpendicular  to  the  wind. 


Finally,  establishment  of  a  wind  direction  Is  not  as  simple  as  it 
sounds.  For  the  purpose  of  fitting,  it  seems  better  to  fit  a  line  through 
the  maximum  dosages  on  the  crosswind  sampling  arcs  and  call  this  the 
•virtual  wind  line",  rather  than  to  rely  on  a  wind  track  obtained  by 
meteorological  observations. 


The  subject  of  fitting  Is  lengthy  and  cannot  be  effectively  discussed 
without  an  actual  example.  It  Is  hoped  that  what  has  been  said  will  serve 
as  an  adequate  introduction  to  the  problem. 
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COMMON  PITFALLS  IN  THE  DESIGN  AND  ANALYSIS  OF  EXPERIMENTS 
Chairman:  G.  E.  P.  Box,  The  University  of  Wisconsin 

Panel  Members:  Cuthbert  Daniel,*  Private  Consultant 

J.  S.  Hunter,  Mathematics  Research  Center, 

The  University  of  Wisconsin 
W.  J.  Youden,  National  Bureau  of  Standards 
Marvin  Zelen,  The  University  of  Maryland 

Prior  to  the  start  of  the  conference,  each  Panel  Member  sent  to  Dr.  Boa 
a  brief  on  the  Common  Pitfalls  in  the  Design  and  Analysis  of  Experiments 
which  he  planned  to  discuss  at  the  Aberdeen  meeting.  We  publish  here  f 
in  outline  form  these  briefs. 


Guthbart  Daniel 

1.  In  design,  the  mistake  I  make  most  often,  is  to  start  planning  experi¬ 
ments  with  insufficient  understanding  of  the  substantive  problem. 

2.  In  analysis,  the  commonest  mistake  I  make  is  to  assume  prematurely 
that  I  know  what  went  on. 

3.  After  these  two  are  out  of  the  way  —  or  forgotten  the  commonest 
defect  in  data  Is  the  presence  of  a  very  .small  number  of  very  bed  values. 
The  pitfall  is  to  fall  to  notico- these  bad  values.  By  a  bad  value  I  mean 
one  whose  observed  or  recorded  magnitude  controls  or  dominates  the  in¬ 
terpretation  of  the  whole  set  of  data. 

4.  In  balanced,  and  especially  in  factorial,  experimentation,  defective 
randomization,  usually  in  the  direction  of  plot-splitting  is  the  commone 
error  of  experimenters  and  Its  presence  undetected  is  then  the  common 
pitfall  In  trying  to  Interpret  the  data. 


1.  Considerable  arithmetical  dexterity  is  required  to  perform  the  “""l**1* 
of  variance  associated  with  many  experimental  designs  and  their  assoda 


♦Mr.  Daniel  was  unable  to  attend  the  moating.  He  telephoned  his  comments 
to  Dr.  F.  E.  Grubbs.  These  were  read  by  Professor  G.  E.  P.  Box. 
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mathematical  models.  Unfortunately,  the  experimenter  frequently  considers 
his  data  analyzed  once  this  arithmetic  Is  completed.  Methods  of  data 
analysis  used  with  great  profit  long  before  the  Invention  of  the  ANOVA  are 
thus  neglected  (graphs,  histograms,  effects  ofdhanges  in  scale,  the  search 
for  abberant  observations,  trends,  etc,).  After  completing  the  regular  ANOVA 
experimenters  also  frequently  fail  to  review  the  residuals  for  additional 
signals  or  to  thtnV.  In  terms  of  alternative  mathematical  models  that  might 
be  expressive  of  the  data. 

2 ,  There  exists  an  unj ustlflable  high  regard  forithe.  ability  of.regresslen 
equations  to  unfold  and  identify  information  within  a  large  accumulation 
of  data.  This  faith  in  multiple  regression  techniques  is  particularly  strong 
whnn  the  data  have  been  haphazardly  collected  with  little  regard  for  either 
randomization  or  good  experimental  design. 


W.  1;  Youden 

My  favorite  "pitfall"  is  elementary,  obvious  and  yet  pretty  treacherous. 
Let  us  suppose  some  ammunition  stored  at  three  temperatures  and  two 
humidities  (6  combinations).  Every  six  months  samples  are  taken  and  fired 
and,  let  us  say,  shell  velocity  determined.  We  have  a  lovely  trap  for  any¬ 
one  who  knows  how  to  do  an  analysis  of  variance.  Suppose  duplicate  firings. 

Rel.  Temp.  Period  stored  -  months 

Hum.  °C  6  12  18  26  ...  . 

0 

SO  20 


mm  m 

etc. 
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presumably  only  ore  storage  chamber  was  available  for  each  storage 
condition.  I  need  not  elaborate  further.  Overlooking  the  fact  that  the 
error  of  split  plot  comparisons  is  usually  less  than  that  for  between  plots 
Is  all  too  common. 


Marvin  Zeien 

Often  non -statistical  pitfalls  may  invalidate  an  entire  experiment  or 
even  cause  Incorrect  conclusions  to,be  made.  No  amount  of  good  statistics 
will  be  able  to  rectify  a  non-statistlcal  blunder.  Three  kinds  of  non- 
statistlcal  pitfalls  discussed  are: 

1.  Analysis  of  data  without  really  understanding  how  the  experiment  was 
executed  may  create  an  incorrect  analysis, 

2.  When  cooperative  experiments  are  being  carried  out  with  groups  not  in 
-  complete  contact  with  one  another,  the  groups  will  often  differ  in  their 

administration  of  treatments  and  evaluation  of  responses. 

3.  Extrapolation  of  data  over  a  different  range  of  experimental  conditions. 


TOMETSSTC  YCSL  OUTLIERS1 


C.  P.  Quesenberry2  and  H.  A.  David 
Virginia  Polytechnic  Institute 

i 

I.  INTRODUCTION.  This  paper  is  concerned  with  the  problem  of  detecting 
outlying  observations  when  In  addition  to  the  normal  sample  x^,  X2, 

xn  at  hand  an  independent  'tnean-squcre  estimate  a2,  of  the  common  variance 

c’2  irr  available.  The  same  situation  hen  been  considered  by  Nair  (1948) . 

To  test  for  one  outlier  at  a  specified  end  of  the  sample  he  proposes  using 
the  ratio  of  the  extreme  deviate  from  the  sample  mean  to  sv  ,  Por  two-sided 
testing  the  extreme  absolute  deviate  from  the  sample  mean  divided  by  sv 
has  been  proposed  by  Halperin,  et  al.  (1955)., 

The  two  statistics  mentioned  above  do  not  make  use  of  the  variance 
estimate  s‘  from  the  sample  and  for  this  reason  do  not  possess  certain 
desirable  optim&l  propetties.  We  shall  propose  statistics  with  the  same 
numerators  os  those  above  but  with  Sy  in  the  denominators  replaced  by  the 
pooled  estimate 

i/a. 

.  Kudo  (1956)  has  shown 


that  among  0  suitably  restricted  class  of  tests  these  statistics  maximise  the 
probability  of  rejecting  the  null  hypothesis  of  homogeneity 'of  the  sample  in 
the  presence  of  a  single  outlier. 

We  shall  develop  a  method  for  computing  percentage  points  of  these 
statistics  and  present,  the  computed  tables.  These  tables  ere  also  immedi¬ 
ately  applicable  to  the  problem  of  slippage  of  means  in  normal  samples. 

Two  examples  illustrate  the  procedures. 

2.  NOTATION  AND  DEFINITIONS.  Let  xj,  xa.  ....  d,9note  th«  sample  in 
the  order  drawn.  Then  define 

*Thts  research  supported  in  part  by  a  National  Science  Foundation  Fellowship 
and  in  part  by  the  Office  of  Ordnance  Research,  U.  S.  Army. 
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Now  at  Montana  State  College. 
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n  n 

_  1  r~>  9  1  C—  —2 

x“  irL  * i  *  5  "  iin  L  <* i-*  * 

t  =  i  i  - 1 

2  2 
s‘  is  an  independent  mean-square  estimate  of  valance,  a*, 

with  V  degrees  of  freedom, 

S2  -  (n  -  1}  s 2  ♦  v  s^  , 

bA  *  -  x)/  S  . 

.b  -  m^x  bt  *  -’i-JBiySL.  X  , 

b*  “  .max  j  bA  j 


(2.1) 

(2.2) 

(2.3) 


Then  b  Bnd  b*  are  essentially  the  one-sided  and  two-sided  statistics, 
respectively,  discussed  in  section  1.  It  should  be  noted  that  S2  has  not  . 
been  divided  by  Its  degrees  of  freedom. 

The  special  case  v  «  0  has  been  treated  by  PearBon  and  Chandra  Sekar 
(193ft),  Grubbs  (1950)  and  Borenlus  (1958). 

3.  DISTRIBUTION  THEORY.  For  the  work  In  later  sections  the  distribution 
of  bj  and  the  Joint  distribution  of  bj  and  bj  are  needed.  We  shall  now 

obtain  these  distributions,  taking  ior  definiteness  l  -  1  and  J  -  2.  An 
extremely  complicated  derivation  of  essentially  *he  same  distributions  has 
been  given  by  Doornbos,  Kesten  and  Prlns  (1956)  in  an  article  concerned 
with  slippage  testa. 

As  is  well  known,  (n  -  l)s  may  be  decomposed  Into  two  independent 
components 


<*1  -  1,2  ♦  Xfr  _  2)  °*  • 


2  2 

which  are  distributed  respectively  as  ^  c*  with  1  and  n  -  2  degree* 


•  * 
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of  freedom.  With  the  same  notation  we  have;  therefore 


t*|  '  *>*  +  ^(n  ♦  v  -  2)  ff‘ 


Then  bj  may  be  written  as 


-JL_  +  7?.  A*  | 

n  - 1  6^  -  x)4  I 


<f2  \  -1/2 


13.1) 


n  -  1 


1/2. 


1  + 


n  ♦  V  -  2 


rV2 


(n  +  y.2)/ 

where  tjn  +  y  _  2)  denotes  a  X“V®riaf*  with  n  +  V  -  2  degrees  of 
freedom.  It  follows  that  the  density  function  of  bj  Is 


f(bj)  - 


>JL.)1/Z.  rl(n  +  y  -  Q/a] 
n“1/  *Vrr[(n+v  -a)/lj  ■ 


1 1  -  nb^  /  (n  -  1 )  j 


1/2  (n  4  V  -4) 


(3.2) 


n  -  1 
n 


1/2 


<=.  b. 


•i*-r 


This  generalization  of  a  result  due  to  Thompson  (1935)  has  also  recently 
been  pa>t?ftt«L  out  by  Anscombe  (I960). 
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Continuing  the  decomposition  of  (3-1)  one  step  further  we  have 


’  lx‘  -;)2  +  K+*-* 


1  V 

x  —h 


Let  fc>2  “  (x2  -  x'  )/S*  ,  - 

where  S'2  -  S2  -  <x.  -  x)2  . 

n  -  1  1 . 


Clearly,  the  distribution  of  ts  of  the  form  (3.2)  with  n  replaced  by 
n  -  1.  Moreover,  b'j  is  Independent  of  bj.  To  888  8uPP°9*  9v  ** 

based  on  a  random  sample  of  size  v  1,  with  mean  x^,  taken  frtxa  a 
N(/^^  ,  a2)  parent-  This  assumption  is  unnecessarily  restrictive  hut  does 
not  essentially  affect  the  argument.  Then  x,  x.and  S_  are  complete 
sufficient  statistics  for  v4*  t-ij  and  a  •  Since  the  distribution  of 
does  not  involve  these  parameters,  b2'  is  independent  of  the  Joint  dis¬ 
tribution  of  x,  Xj  and  S  (Basu,  1955T-  Also  does  not  involve  x, 

so  that  it  must  be  independent  of  b^. 

The  Joint  density  function  of  bj  and  b‘2  is  thereto* 

*  (‘-*7*  »;2)  . 


Design  of  Experiment* 


<1  n  -1  ‘  1/2 


1 


-  .1/2 

is-i-M  ^  bn  ± 

n'  -  1 


*  ^  n  -  2 

V»  "o  11 1 

4  n  -  1 


1/2 


Since  b  ■ 


b2  +  b|/  (n  -  1) 


2  r  2  ...1  1/2 


[' 


-nb‘/(n-l)*j 


we  obtain 


...  .  |  n  \I/Z  (n  ♦  v  -  3) 

f{bL'  b2*  ~  [  „  -  2  )  2  X 


x  1- 


n  - 1  2  2bj  b2 


n  -  2  _  bl  n  -  2 


n  - 1 
n  -  2 


|  (1/2)  (n  +  V  -5) 


over  the  ellipse 


2S2 
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f  (bj. .  b2)  =  0,  elsewhere. 

Moments  of  b  and  b* 

2 

Since  the  distributions  of  b  andjb^.do  not  involve  p,  M-j,  and  O  it 
follows  as  above  mat  b  and  b*  are  distributed  Independently  of  S.  This 
result  is  well  known  for  the  special  case  v  *  0  and  may  indeed  be  proved 
in  a  similar  fashion,  as  was  pointed  out  to  us  by  Dr.  G.  E.  P.  Box.  As  a 
consequence  of  this  independence  the  moments  {about  zero)  of  b  and  b*  »re 
the  ratios  of  the  moments  of  their  respective  numerators  and  denominators . 
Thus  we  have,  for  example. 


£<br> 


max  ~ 
£  Sr 


!n  this  case  (but  net  so  readily  for  b*)  the  right  hand  side  can  be  evaluated 
numerically  since  the  cumulants  of  *na)C  "  «  are  related  to  the  tabulated 

cumulants  of  the  extreme  (Ruben.  1954)  by  equations  which  for  \l  -  0.  a  •  1 
become  (McKay,  1935) 

=<r{xma*)  r-1.  3,  4,  5,  ... 

x  niox  i  nwx 


*  2  »  -h  <’W>  -  ‘/n- 


The  distribution  of  b  may  therefore  be  approximated  by  a  Pearson  Type  curve 
However,  for  the  purpose  of  obtaining  upper  percentage  points  the  approach 
of  the  following  section,  applicable  to  both  b  and  jj*.  is  preferable. 

4.  THE  COMPUTATIONM,  PROCEDURES^ 

(a)  One-Sided  Case,  b 

We  now  consider  a  procedure  for  computing  significance  points  of  b 
defined  by  (2.2).  For  a  given  value  of  j}  and  V  let  D*  be  the  required 
significance  point  of^.  By  Bonferronl’s  inequalities  (cf.  David,  1956) 
we  have  for  any  D 
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#Pr{*j>D)  -  (?)  PrO^D).  (^.l) 

For  sufficiently  large  D  the  right  side  Befves  as  n  first  approximation  to 
Pr(b.>D)  and  the  left  aide  as  a  second  approximation.  If  the  first  ap-- 
proximatlon  is  set  equal  to  ©b  then  the  resulting  equation,  i.e. 


Pr  Cbx  >  D)  «  o/n 


cen  be  solved  for  a  value  D^,  which  is  an  upper  bound  of  the  value .  . 

sought.  From  (3.2)  this  equation  can  be  written  aa 


°yn  -  c 


n(l/2)(n+-v-4) 
bf  dbj 


where  C 


.  r  n  1 
•  I  7c(n  -  1)  j 


(n  ♦  -y  -1) 


p[(n  +  -v- 2)/2j 


The  folio’ll ng  equivalent  equation  ia  more  convenient  to  work  with 


-v-41/2 


♦  f  j  [  n  fv  -  4]  [n  +v  -6]  . . .  [n  ♦  -v>-2_(r  +1)1  tipf  T  .  ^  2) 
,*2*1  r  1  *  2r  •  In  -  l)r  •  C2r  +  1)  ,  .  ' 
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where 
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a,  .  -as*  . 

1  °i 


3y  transposing  the  second  term  on  thp  right  to: the  left  of  (4.2)  the 
equation  can  be  identified  with 


Vi -i  ’ h 


b  •  •  -  •) 


so  that  Newton's  iterative  formula 


Dl,l  “  Dl.  i  -  1 


"(V'-i)' 


can  be  used  to  solve  for  Dj.  Note  also 


„•  .  [l  -  D,2 


i(l/2'Xn  +  -y  -4) 


The  initial  value  of  DA  used  to  start  the  iteration  procedure  was  Djo“  ®|« 
as  given  above. 

While  Dj  is  an  upper  bound  for  ,  a  lower  bound  by  (4.1)  satisfies 

nPrlbj^Dj)  -  (2  )  pr0>j  >  D2»  bJ  ^  ^  * 

A  first  approximation  D2  Q  to  D2  is,  therefore,  given  by 

jn  \ 

nPr(bj  >  D2  g)  -  J  Prtb4>  Dj.  hj  > 


H. 
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On  replacing  Di  in  (4.3)  by  D2  g  a  second  approximation  D2  j  is  ob¬ 
tained;  The  process  can  be  continued  until  j  +  \  and  Dj^  t  agree 
to  three  decimal  places.  In  the  present  cate  Dj  g  was  found  to  be  suf¬ 
ficiently  accurate  in  all  but  a  few  cases. 

The  second  term  on  the  right  side  of  equation  (4.3)  is  evaluated  by 
numerical  integration.  The  joint  density  f(b^,  b^)  given  by  (3.3)  is  in¬ 
tegrated  over  the  region  for  which  bj  >  Dj  and  bj  ^  Dj.  ThU  region  is 

shaded  in  Fig.  1.  This  numerical  integration  was  performed  on  an  I.B.M. 

650  computer.  The  numerical  method  used  is  equivalent  to  fitting  an  in¬ 
creasing  number  of  planes  to  the  density  surface  until  the  desired  accuracy 
is  achieved. 

An  examination  of  Fig.  1  shows  that  if  D^>  [jn  -  2)/2n J  ^ 
then  the  second  term  on  the  right  side  of  -(4.3)  is  stero.  Then  dj  •  Dj  •  Dot, 

is  the  exact  percentage  point  of  b.  This  is  important  in  that  it  allows  the 
exact  calculation  of  a  number  of  percentage  points  for  lower  values  of  n 
and  2L- 

The  lower  and  upper  bounds  for  the  percentage  points  were  found,  to  agree 
so  well  for  the  values  of  ot  considered  here  (.01,  .05)  that  only  one  value 
had  to  be  tabulated.  Tables  1  and  2  give  the  1  and  5  per  cent  points, 
respectively,  for  selected  values  of  ^  and  n. 

(b)  The  Two-Sided  Case,  b* 

Essentially  the  same  procedure  is  used  to  obtain  the  significance  points 
of  b*  as  for  b.  The  Bonferroni  inequalities  in  this  case  give 

nPr(  |bjJ>D)  -(^)  Pr(  |  >13,  J'jj j?D)i  Pr(b*?OK  hPr{  1  b^b). 

From  the  symmetry  of  f(b|)  we  have 

Pr(Jbtl>D)  «  2Pr  (b±  >  D). 


Let  be  the  desired  significance  point  cf  b*.  Bhut  an  upper  bound 
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Dj*  con  be  obtained  from  (4.2)  by  replacing  OC  by  OT/2  In  ^  and 
solving.  A  flrstf  approximation  D*  to  a  lower  bound  D%  onD^,  Is 
given  by  '■ 


nPr(b,?D^0l 


vJ: 


(4.5) 


A  second  approximation  D*  can  be  obtained  by  replacing .  D^|  by  q 

In  (4.5),  etc.  The  second  term  on  the  right  of  (4.5)  is  evaluated  this  time 
by  integrating  f(bj,b2)  over  the  area  in  each  quadrant  where  J  bj  >  D^  and 

|  b yl^&f  •  The  bounds  on  do  not  agree  aowell  as  for  the  one-sided 

case.  Tables  3  and  4  give  bounds  for  DJ  for  (X  "  .01  and  pf  ■  .05,  res¬ 
pectively.  When  the  bounds  agree  to  three  places  only  one  value  Is 
tabulated. 


5*  THS.sy ?PAttE  PROBLEM .  The  statistics  £  and  |>*  are  rueful  In  treating 
the  slippage  problem  for  normal  populations.  Mere  we  have  the  sample 


We  wish  to  test  the  hypothesis  that  the  entire  array  is  from  a  common  normal 
parent  against  the  alternative  that  the  1  th_  sample  (x^j,  *12'  '***  *in  ^ 


is  from  a  normal  parent  with  a  different  mean,  where  1_  Is  unspecified. 
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‘H 


2 

and  s  be  en  independent  mean  -square  estimate  of  error  with  t  degrees 
of  freedom. 

An  important  special  case  is  that  of  all  equal  subsample. sixes,  i,a.  nj  «* 
n2  *  ...  «  n2  *  m.  For  this  special  case  the  statistics 

Vm"“ (x.  -  3  ' 

max  - - -  ,  (5.2) 

1  ■  S 

1  " 


and 


i  -  1  \  "  1 


are  distributed  as  b  and  ,  respectively.  The  significance  points  of 
these  statistics  are  obtained  from  the  tables  of  b  and  fej..  with  the  para¬ 
meters  n_  and  ■y,  of  the  tables  replaced  by  n"k  and  V  “  k  (m  -  1 )  *  t 
These  slippage  tests  possess  the  same  desirable  properties  as  do  outlier 
tests  based  on  b  and  b*  (see  Paulson,  1952}. 
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If  the  sample  sizes  nt  ere  not  ell  equal  but  bre  approximately  so  tho 
tables  of  b  and  b*  can  be  used  to  obtain  approximate  tests. 

Put 


Then  the  statistics 

max 


Vn4.  x. 


n.  x.  -  x 
1.  I  w 


(5.5) 


(5.6) 


give  approximate  tests  based  on  the  tables  of  b_  and  £*.  The  parameters 
H  and  of  the  tables  are  here  n  ■  k  and  -v  ■  N  +  t  -  k. 

6.  EXAMPLES.  We  no w  give  two  examples  to  illustrate  the  use  of  the  tablet. 
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.  Example  l. 

Squibs  are  small  devices  for  igniting  the  rocket  .motors  of  missiles. 

Wa  tertlghtr.es  s  and  shock  resistance  are  important  characteristics  of  aquibp. 

In  order  to  study  these  characteristics  of  a  large  batch  a  random  sample  of 
size  48  was  drawn.  The  sample  was  randomly  subdivided  into  3  equal 
groups.  The  first  group  was  used  as  a  control  unit  and  received  no  treat¬ 
ment,  the  second  group  was  submerged  in  water  and  the  third  group  was 
dropped  from  a  fixed  height;.  Each  squib  in  the  entire  sample  was  tested  by 
having  a  current  of  5  amperes  passed  through  it  and  its  time  to  failure 
recorded. 

From  previous  experience  it  is  felt  that  these  delay  timft3  are  epprord. merely 
normally  distributed.  It  Is  also  known  from  previous  experience  that  occas¬ 
ionally  extremely  large  delay  times  occur.  Because  of  this  an  outliers  test 
was  used  on  each  subgroup  to  guard  against  such  spurious  observations. 

The  variance  was  assumed  to  be  constant  throughout  the  experiment.  The 
data  are  given  in  Table  6.1. 

First  we  test  each  of  the  subgroups  for  outlying  observations.  For  each 
test  the  variance  estimates  from  the  two  remaining  subgroups  are  used  as  an 
independent  estimate  of  error.  For  the  control  group  we  heve  from  (2.1) 

•  \  ■  -  ' 


b 


.76  -  .4438 
■\C8698"  . 


“  .3390  . 


Table  2  gives  the  5  per  cent  point  for  n  -  16  and  v  ■  30  as  app.  md- 
mately  b  (  .OS:  16,  30  )  -  .384,  <so  the  above  valus  does  not  attain 
significance. 

For  the  waterrtightness  group 


b  -  -At?.? . -  .  -  .5124 

.93274 


and  this  is  compared  with  the  same  value,  b  (  .05;  16,  30  )  ■  .384,  me 
before.  This  is  highly  significant  and  the  observation  1.09  la  rejected. 


Table  6.1 


Control  (x^) 

Watertigbtaese  (>21) 

•  38 

03 

.26 

•33 

.41 

•  30 

•  33 

•43 

•  33 

1.09 

.37 

.  .46 

1  .34 

07 

•76 

•47 

•51 

•39 

03 

•74  . 

03 

•  32 

.41 

.74  . 

•47 

.48 

.49 

•37  . 

.42 

1  *5® 

.34 

.44 

L _  ..... 

Bbock 


(Data  furnished  by  Ordnance  Missile  Laboratories,  AROMA,  AOHC 
Redstone  Arsenal,  Alabama.) 

E-u  ■ va  I*2i  ■  e'»  E*»  ■  7•^, 

Sj.  -  .4430  Xg  «  .5186  -  A556 

jT*^  -  3.3606  £x^  -  4.6768  £,8,  -  5.40a 

Io  T5~  15" 

88(1)  -  .2100  89(2)  «  .5712  89(5)  .  .Ofln 


7  /  -  4.3056 
16 

89(2)  «  .5712 


88(5)  -  .Oft) 
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New,  since  the  1.09  observation  In  the  2nd  group  was  unduly  In¬ 
flating  the  variance  estimate  for  the  first  test,  we  shall  recompute  that 
test  with  this  observation  omitted  from  the  variance  estimate.  The  test 
statistic  becomes 


.76  -  .4438 
V.52T7” 


.4378  , 


and  13  to  be  compared  with  b  {  .05;  15,  29  ).  Table  2  gives  b  (  .05; 

15,  7.4)  *.  .413  and  increasing  either .  n  or  -y  tends  to  decrease  the 
percentage)  point,  so  the  above  statistic  is  significant.  The  observation 
.76  is  omitted  from  the  control'  group.  The  sum  of  squares  based  on  the 
remaining  15  observations  is  .1113.  4 

For  testing  the  3rd  (shock)  group  b  »  .2707.  This  la  not  significant 
at  the  ,05  level.  Further  tests  In  the  subsamples  lead  to  no  more  dis¬ 
carded  observations. 

The  purpose  of  the  experiment  is  to  test  the  significance  of  the  water 
and  shock' treatments.  We  are  interested  in  testing  the  hypothesis  that 
either  one  or  both  treotments  increased  the  mean  delay  time.  A  two-sided 
test  Is  appropriate  here.  For  the  two-sided  test  will  have  a  probability  of 
rejection  higher  than  for  the  null  situation  under  any  alternative  except 
when  one  treatment  effect  is  exactly,  twice  the  other  (both  non-zero).  . 

Since  the  subsample  sizes  ere  large  and  nearly  equal,  the  approximate 
test  discussed  in  section  5  should  give  accurate  results.  Here  nj  ■ 

-  15  and  n3  -  16.  The  weighted  mean, s  are  Vn^  •  1.637,Vnj,X2  * 
1.862,  V«3  x3  «  1.822  and  -  1.774,  Then  (5.6)  gives 


1,774  -  1,637 
.6442 


0.213 


This  is  to  bo  compared  with  b  (  ,05;  3,  43  ).  Table  4  gives  b  (  .05; 
3,  40)  2:  0.283  so  the  value  0.213  is  not  significant  at  the  .05 
level.  V/o  conclude  that  the  treatments  hod  no  effect. 
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Example  2. 

A  sample  of  six  observations  was  drawn  from  a  table  of  random  normal 
numbers  and  a  randomly  selected  observation  was  increased  by  two  standard 
deviations.  .The  observations  obtained  were  265,  7.23,  291,  105,  43 
and  477.  A  sample  of  six  observations  was  drawn  from  a  table  with  the 
sura  a  variance  but  with  a  different  mean  to  give  an  independent  estimate 
of  variance.  These  observations  were  171,  111,  165,  271,  58  and  217.  ' 
The  moan  and  sum  of  squares  about  the  mean  for  the  first  set  of  observations 
are  234  and  116,504  respectively.  The  sum  of  squares  about  the  mean 
for  the  second  set  is  26,519.  So  the  one  sided  test  statistic  Is  from  (2.1) 


b  _  477  -  234 
\  143,023 


.643. 


Table  2  gives  b  (  .05;  6,  5  )  •  .638,  so  that  the  observation  477  Is 
rejected  at  the  .05  level.  To  bio  4  gives  b*  (  .05;  6,  5  )  ■  .681,  so 
that  the  observation  477  Is  not  rejected  at  the  .05  level  by  the  two- 
sided  test. 
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APPENDIX 
THE  TABLES 

Tables  1  and.  2  give  the  1  and  5  per  cent  points,  respectively,  of 
b  [pee  (2.1)^).  The  1  per  cent  points  are  correct  to  1  unit  in  the 
fourth  place.  Only  a  few  values  for  n  and  v_  large  are  questionable  at 
all  in  the  last  digit.  The  five  per  cent  points  of  b  are  correct  to  three .. 
places  except  for  a  few  large  values  of  n  and  For  n  ■  20  and  V“  50 
the  value  given  may  be  is  such  as  2  units  large  in.  the  third  place.  Mo 
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other  values  in  the  5  per  cent  table  are  incorrect  by  more  than  one  unit  in 

the  third  place. 

Tables  3  and  4  give  lower  and  upper  bounds  for  the  percentage  points 
oi  b*  fcee  U.2)*]  .  For  each  combination  of  parameter  values  lower  and 
upper  bounds  are  given  except  when  these  bounds  agree  to  three  decimal 
places  and  then  only  one  value  is  given.  .  • 


The  upper 


I  %  points  of  ‘  x  )/S  for 


~v+l 


s2  “  2^  ^  *■  *)Z  +  ^  ~  * Bnd  y*  lndependent  of  xj 


Table  1' .  .  . 


The  upper 


5  %  points  of  (  -  x  )/S  for  • 


S 


n  V+i 

2  ^  —.2 


-Z*- 


-.2 


■J)  +  ^  (y^  -  y  )  •  and  y*  Independent  of  Xj, 

1  1 


Table  2 


The  upper  1  *  points  of  max  |xt  -  x  |/S  for 


S‘ 


n  + 1 

5'ta-a1*  V  (yA  -  y)2  ,  and  y^  independent  of  Xj, 

x  <V 


Table  3 
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Table  1 

Table  of  points  of  the  distribution  of  b 


gsa 

Eb 

3 

5 

6 

7 

8 

B 

O.S165 

0.8617 

0.8739 

0.8695 

O.8566 

0.8394 

o.am 

0.8431 

0.8478 

O.84OO 

0.8263 

0.8104 

0.7904 

0,8155 

0.8176 

0.8094 

0.7971 

0.7833 

0.7614 

0.7844 

O.7865 

0.7800 

0.7698 

0.7579 

B 

0.7299 

0.7532 

0.7570 

0.7527 

0.7444 

0.7341 

5 

0.6990 

0.7238  . 

0.7297 

0.7274 

0.7207 

0.7120 

0.6703 

O.6968 

0.7045 

0.7037 

0.6987 

0.6918 

B1 

0.6442 

0.6720 

0.6812 

0.6819 

0.6786 

0.6729 

a 

0.6204 

0.6491 

0.6597 

0.6620 

0.6599 

0.6554 

9 

0.5986 

0.6282 

0.6401 

0.6436 

O.6424 

0.6389 

10 

0.5788 

0.6091 

0.6219 

0.6263 

0.6263 

0.6237 

12 

0.5441 

0.5723 

0*5895 

0.5956 

0.5971 

0.5962 

15 

0.5017 

0.5333 

0.5489 

0.5566 

0.5600 

0.5607 

20 

0.4482 

0.4795 

0.4962 

0,5055 

0.5106 

0.5132 

21. 

0.4158 

O.4463 

0.4633 

0.4732 

0.4792 

0.4826 

30 

0.3779 

0.4073 

O.424O 

0.4346 

0.4412 

0.4455 

40 

0.3328 

0.3600 

0.3763 

0.3869 

0.3940 

0.3990 

50 

0.3006 

0.3261 

0.3416 

0.3519 

0.3591 

0.3642 
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10 

12 

15 

20 

0.3211 

0.3032 

0.7687 

0.7228 

0.6614 

0.7942 

0.7780 

0.7465 

0.7048 

0.6483 

0.7638 

0.7541 

0.7260 

0,6879 

0.6356 

0.7450 

0.7320 

0.7070 

0.6723 

0.6239 

0.7229 

0.7116 

0.6890 

0.6576 

0.6127 

0,7026 

0.6926 

0.6724 

0.6438 

0.6020 

0.6337 

0.6748 

0.6548 

0*6306 

0.5917 

0,6659 

0.6581 

0.6422 

0.6182 

0.5622 

0.6495 

0.6428 

0.6286 

0.6066 

•0.5727 

0.6341 

0.6234 

0.6156 

0.5956 

0.5638 

0.6198 

0.6148 

0.6631 

0.5651 

0.5556 

0.5935 

0.5899 

0-5808 

0.5654 

0.5398 

0.5597 

0.5576 

0.5513 

0.5393 

0.5183 

0.5140 

0.5136 

0.5104 

0.5031 

O.488O 

0.4844 

0.4850 

0.4837 

0.4785 

O.4668 

0.4480 

O.4496 

0.4501 

0.4479 

0.4393 

0,4023 

0.4047 

0.4071 

0.4074 

0.4038 

0.3681 

0.3711 

0.3744 

0.3766 

0.3751 

Table  of 


__3 _ 

0.8154 

0.789 

0.7U 

0.692 

0.648 

0.610 

0.577 

0.549 

0-524 

0.502 

0.483 

0.450 

0.U1 

0.363 

0.335 

0.303 

0.266 

0.239 


Table  2 


555  points  of  the  distribution  of  b 


4 

5 

6 

mm 

8  J 

O.844 

0.836 

0.815 

0.791 

0.768 

0.800 

0.789 

0.771 

0.752 

0.733 

0.752 

0.745 

0.732 

0.717 

0.701 

0.707 

0.705 

0.697 

0.686 

0.673 

0.668 

0.671 

0.666 

0.658 

O.648 

0.634 

0.640 

0.638 

0.633 

0.625 

0.604 

.  0.613 

0.614 

0.610 

0.604 

0.576 

0.589 

0.591 

0,590 

0.586 

0.554 

O.567 

0.571 

0.571 

0.569 

0.533 

0.547 

0.553 

0.554 

0.553 

0.515 

0.530 

0.536 

0.539 

0.538 

0.462 

0.499 

0.507 

0.511 

0.512 

0.443 

0.461 

0.471 

0.476 

0.479 

0.395 

0.413 

O.425 

0.431 

0.436 

0.366 

O.384 

0.396 

0.403 

O.408 

0.332 

0.350 

0.362 

0.370 

0.375 

0.292 

0.309 

0.321 

0.329 

0.335 

0.264 

0.280 

0.291 

0.299 

0.305 

Table  2 

(continued) 

• 

iV 

9 

10 

12 

15 

20 

0*746 

0.725 

0.689 

O.644 

0.586 

0*714 

0.697 

0.666 

0.626 

0.574 

0.666 

0.672 

•  O.644 

0.609 

0.562 

0.661 

0.648 

0.625 

0.594 

0.550 

0.638 

0.627 

0.607 

0.579 

0.540 

0.617 

O.606 

0.591 

0.566 

0.530 

0.598 

0.591 

0.575 

0.553 

0.520 

0.580 

0.574 

0.561 

0.5U 

0.511 

O.564 

0.559 

0.546 

0.530 

0.502 

0.550 

0.546 

0.536 

0.520 

0.494 

0.536 

0.533 

0.524 

0.510 

0.486 

0.511 

0.509 

0.503 

0.492 

0.472 

O.46O 

0.479 

0.476 

0.466 

0.452 

0.438 

0.439 

0.439 

0.435 

0.424 

0.411 

0.413 

0.415 

0.413 

0.405 

0.379 

0.362 

0.385 

0.385 

0.361 

0.339 

0.342 

0.347 

0.349 

0.349 

0.310 

0.313 

0.318 

0.322 

0.323 

Table  3 

Table  of  1/6  points  of  the  distribution  of  b+ 


D 

0.6165 

O.864 

* 

0.881 

* 

0.882 

# 

0.874 

# 

0.660 

* 

KS 

0.614 

* 

0.851 

* 

0.862 

* 

O.858 

* 

0.847 

# 

0.833 

♦ 

2 

0.600 

* 

0.830 

# 

0.837 

* 

0.631 

* 

0.821 

* 

0.608 

* 

3 

0.778 

A 

0.605 

* 

0.809 

* 

0.805 

* 

0.796 
'  * 

0.785 

* 

D 

0.751 

* 

0.777 

* 

0.782 

♦ 

0.779 

* 

0.772 

* 

0.762 

* 

5 

0.724 

* 

0.750 

* 

0.757 

* 

0.755 

* 

0.749 

* 

0.7U 

* 

6 

0.698 

0.725 

* 

0.733 

0.733 

* 

0.728 

* 

0.721 

* 

0,651 


0.629 


0.576 


vm 


0.323 

0.317 


0.702 


0.680 


0.659 


0.640 


0.607 


0.564 


0.509 


0.711 


0.690 


0.671 


0.653 


0.621 


0.580 


0.526 


0.434  0.451 


0.401 


0.34?  0.365 

0.31.6  0.363 


0.712  0.708 


0.692  0.690 


0.674  0.673 


0.657  0.657 


0.626  0.628 


0.587  0.590 


0.534  0.539 


0.507 


O.46I  0.468 


0.411  0.41« 


0.375  0.382 

0,373 


0.703 


0.665 


0.669 


0.626 


0.541 


0.510 


0.38 

0.38 


9 

10 

12 

15 

20 

0.843 

* 

0.827 

* 

0.794 

♦ 

0.750 

* 

0.666 

■0.619 
t  * 

0.804 

# 

0.773 

# 

0.732 

* 

0.674 

0.795 

* 

0.781 

* 

0.753 

* 

0.715 

* 

0.662 

0.772 

* 

0.759 

* 

0.735 

* 

0.700 

0.651 

0.751 

* 

0.740 

# 

0.717 

* 

0.636 

0.639 

0.731 

* 

0.721 

♦ 

0.701 

0.671 

0.628 

‘  0.713 
+ 

0.704 

0.685 

0.658 

■  * 

0.617 

0.695 


0.679 


0.664 


9 


0.623 


0.53$ 


0,542 


0.511 


0.687 


0.672 


0.657 


0.644 


0.619 


0.586 


0.541 


0.511 


0.671 


0.657 


0.644 


0.632 


0.609 


0.579 


0.537 


0.509 


.646 


0.634 


0.623 


0.612 


0.593 


0.566 


0.523 


0.503 


0.608 


0.599 


0.591 


0-581 


0.565 


0.544 


0.512 


0.490 
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Table  of  5$  points  of  the  distribution  of  b* 
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0.373 

0.359 


0.600 
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0.579 


O.560 


0.526 


0.436 


0*434 


0.403 

0.395 


0.611 

0.610 


0.592 


0.573 


0.542 


0.502 


0.452 


0.421 

0.L16 


0.635 


0.614 


0.596 


0.579 


0.511 


0.462 


0.432 

0./.28 


0.632 


0.613 


0.596 


0.530 


0.515 


0.466 


0.367  0.334 


B 

0.6162 

0.855 

* 

0.857 

* 

0.844 
~  * 

0.825 

* 

0.804 

♦ 

Hi 

0.803 

* 

0.324 

* 

0.320 

* 

0.807 

♦ 

0.789 

* 

0.771 

* 

2 

0.769 

* 

•  0.786 

* 

0.782 

* 

0.771 
•  * 

0.757 

* 

0.741 

* 

3 

0.729 

* 

0.747 

* 

0.746 

* 

0.738 

* 

0.727 
.  * 

0.714 
♦  • 

II 

0.690 

0.711 

* 

0.713 

* 

0.708 

* 

0. 670 

0.689 

5 

0.655 

0,654  _ 

0.673 

O.684 

0.681* 

0.675 

0.667 

g 

0.623 

0.620 

0.649 

0.657 

0.657 

0.652 

0.646 

0.627 


0.609 


0.579 


0.433  0.442 
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0.403  0.403 


0.340 

0.351 

0.359 

O.364 

J2J&L. 

B'W  HmBqBjI 

■a«ai 

0.308 

0.319 

0.326 

0.332 
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12 

15 

20 

0.733 

0.763 

0.727  • 

0.681 

0.621 

* 

* 

•  * 

O.J53 

O.J36 

0.704 

0.663 

0.608 

O.J26 

O.jU  • 

0.633 

O.646 

0.596 

0.701 

0.633 

O.664 

0.630 

O.584 

0.6?3 

0.667 

0.646 

0,6l6 

0.573 

0.6  53 

0.643 

0.629 

0.602 

0.563 

0.633 

0.630 

0.613 

0.589 

0.553 

0.621 

0.614 

0.599 

0.577 

0.544 

0.604 

0.598 

O.586 

0.566 

0.535 

0.539 

0.534 

0.573 

0.555 

0.526 

0.575 

0.571 

0.561 

0.545 

0.518 

NOTE  ON  PRECISION  OF  GRADED  TS. 

ALL-CR-NONE  RESPONSE  IN  BIOASSAY 

Francis  Marlon  Wad  ley 

U.  S.  Army  Chemical  Corps  Biological  Laboratories 

For  purposes  of  biological  assay,  two  types  of  response  are  generally 
available.  The  graded  response  furnishes  for  each  subject  a  measure  of 
effect,  while  the  "quanta!"  or  all-or-none  response  provides  for  each  subject 
merely  the  fact  that  It  did  or  did  not  respond.  In  the  latter  case.  In  order 
to  obtain  an  arithmetic  figure  for  analysis,  it  is  necessary  to  use  a  number 
of  subjects  and  to  record  the  proportion  responding  to  a  given  stimulus. 

This  Is  the  procedure  of  dosage-mortality  studies;  a  single  test  Involves 
determination  of  proportions  responding  to  several  concentrations  of  the 
test  material. 

The  graded  response  is  attractive  because  It  provldee-e  definite  measure 
of  response  for  each  subject,  and  because  It  has  a  simple  relationship  with 
basic  regression  analysis.  Since  the  measure  of  extent  of  an  effect  Is  more 
precise  than  the  mere  statement  that  it  passed  or  did  not  pass  »  certain  point, 
a  successful  graded  response  obviously  gives  more  information  per  subject 
than  an  all-or-none  response.  This  added  precision,  with  more  efficiency 
in  expensive  experimentation,  is  of  considerable  Importance  to  biologists. 

It  frequently  is  true  that  a  good  graded  response  Is  not  available,  or 
that  the  all-or-none  response  is  the  pnly  one  that  will  answer  the  question 
at  issue.  However,  It  Is  Important  to  keep  both  possibilities  in  mind  in 
the  choice  of  «n  experimental  program.  To  be  of  maximum  usefulness,  a 
graded  response  must  show  a  consistent  and  strong  relationship  with  the 
concentration  of  the  material  tested,  and  all  subjects  must  respond  to  some 
extent. 

A  well-established  method  of  treating  all-or-none  data  Is  the  log-problt 
method,  which  has  considerable  evidence  of  validity.  It  assumes  a  normal 
distribution  of  the  logarithms  of  tolerance  or  susceptibility  among  individual 
subjects.  Finney  (195Za)  states  that  the  mean  of  this  normal  distribution  is 
estimated  by  the  Ioq  of  the  LDS0  or  n;;  the  standard  deviation  by  the  re¬ 
ciprocal  of  the  problt  regression  coefficient,  1/b. 

If  it  were  possible,  for  the  individual  subjects  in  a  problt  test,  to  read 
individual  log  tolerances  directly,  the  variance  of  log  tolerances  would  be 
given  by  1/br;  the  variance  of  m,  the  mean,  by  1/trn. 


CO 


Va  -  l/b1© 


2  BO 


Design  of  Experiments 

A  successful  graded  response  must  obviously  have  a  high  correlation  with 
log  tolerance;  and  if  the  log  tolerance  would  be  read  directly,  it  would  con¬ 
stitute  an  excellent  graded  response  with  mean  and  variance  as  stated.  For 
the  probit  solution,  where  x.  is  log  concentration  and  n.w  is  the  probit 
weight,  the  variance  of  jn  .Is  estimated  as: 

(2)  _  '  Vm  “  1/b®1  £  l/.Z  nw  +  (m  -  x)*  /T.  nw  (x  -  x)2  [j 

With  good  choice  of  concentrations,  the  second  term  in  brackets  is  often 
negligible  when  data  are  well  balanced  around  50  percent. 

the  two  expressions  for  variance  may  serve  to  give  a  preliminary  com¬ 
parison  of  precision  in  graded  and  all-or-none  response.  For  the  problt, 

(2)  may  be  simplified  to  1  /  £  nwb*  to  compare  with  l/nb.  Since 

Finney's  "So"  or  ][]n  is  equivalent  to  the  "n"  of  (1),  the  differing 
factor  in  \v,  the  average  weighting  coefficient  used  in  (2).  This  w.  may 
average  0.5  in  a  problt  solution  with  well-spaced  concentrations,  but  is 
more  often  a  little  lower.  Thus  the  comparison  of  l/nb*  with  1/nwb* 
chows  the  graded  response  with  a  veriance  a  little  less  than  half  the  var- 
lance  for  the  all-or-none  response.  It  indicates  that  a  good  graded  res¬ 
ponse  may  make  about  twico  os  efficient  a  use  of  subjects  as  a  good  quantal 
response.  This  relation  was  brought  out  by  Gaddum  (1933). 

To  attempt  tests  of  this  result  with  actual  data,  it  is  necessary  to  re¬ 
late  variances  of  estimates  by  the  two  methods.  Hewlett  and  Plackett 
(1956)  compare  1/b  (all-or-none)  with  s/b^  (graded  response),  where 

b^  is  regression  of  graded  response  on  concentration  and  b  Is  the  standard 

deviation  from  regression.  They  tabulate  about  50  values  of  each  from 
the  literature  and  show  that  1/b  and  s/bg  have  similar  means  and  ranges. 

The  quantity  s/bg  Is  the  basis  of  error  calculations  with  graded  response 
(Finney  1952b).  Hewlett  and  Plackett  cite  these  sots  oi  data  from  verte¬ 
brate  subjects;  the  sets,  of  course,  were  unpaired.  It  may  be  assumed 
that  the  responses  were  well  adapted  or  else  they,  would  not  have  been 
published. 

In  this  study,  for  a  more  direct  comparison,  several  sets  of  graded 
response  data  were  adapted  to  oll-ar-none  study.  A  particular  level  of 
response  was  defined  as  "critttnaU",  and  for  each  dose  level  the  percentage 
of  subjects  reaching  or  falling  to  reach  this  "critical"  level  was  determined. 
The  critical  value  was  defined  so  as  to  bo  near  the  mean  and  to  provide  a 
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usable  series  of  percentages.  The  variance  of  log  LD^g,  from  probit  an¬ 
alysis  of  these  percentages,  is  compared  with  the  variance  of  log  concen¬ 
tration  needed  for  the  critical  graded  response,  as  defined  by  regression. 

The  first  set  of  data  was  taken  from  an  article  by  the  writer  (Wadley 
1949,  Table  4).  Guinea  pigs  were  the  subjects,  and  diameter  of  the  ir- 
ritated  area  after  tuberculin  injection  was  the  response.  Three  lots  of  tub¬ 
erculin  were  tested  at  each  of  3  concentrations  at  10 -fold  intervals.  Each 
concentration  of  eech  lot  v/as  injected  into  4  guinea  pigs.  The  3  lots 
were  quite  similar  in  potency,  and  only  the  various  concentrations  produced 
significant  differences  in  the  level  of  response.  Heoce  the  36  observations 
were  grouped  into  12  tuberculin  reactions  In  each  of  3  concentrations. 

The  "critical*  response  level  was  defined  as  an  Irritated  area  with  a  diameter 
of  12  millimeters.  In  the  3  groups  of  animals  this  level  was  reached  by  8.3 
per  cent.  66.7  per  cent  and  100  per  cent  for  the  low,  medium  and  high 
concentrations,  respectively,  giving  a  basis  for  probit  analysis. 

The  value  iocnd  for  s/b  was  0.41,  while  1/b  was  estimated  at  0.36, 
a  close  correspondence.  The  variance  of  m.  was  estimated  as  0.008  for 
graded,  0.013  for  all-or-none. 

A  second  set  of  data  is  from  Finney  (1952b,  Table  9.1)  on  weight  gaips 
of  rats  following  vitamin  doses,  with  10  responses  at  each  of  3  concentra¬ 
tions.  The  critical  response  Indicated  was  36  units,  which  gave  a  percent¬ 
age  series  of  10,  70  and  80  per  cent.  Two  more  sets  from  Fort  Detrick 
data  Involved  the  exposure  of  guinea  pigs  to  toxic  aerosols.  Log  of  survival 
time  in  hours  was  the  response.  The  critical  response  was  taken  as  the 
mean  log  (about  2.00);  one  percentage  series  was  25,  38,  94;  the  other 
was  7,  12,  31,  69,  100.  These  were  used  in  analysis  as  with  the  tub¬ 
erculin  data.  Results  ore  brought  together  In  a  table  below. 


Comparison  of  All-or-None  with  Graded  Response 


Problt  Results 

Graded  Response 

Yro  Ratio 

Response 

iZk 

Ytn 

s/bq 

Vm  ' 

Problt/Graded 

Tuberculin  reaction 

0.36 

0.013 

0.41 

0.008 

1.62 

Weight  gain 

0.22 

0.0031 

0.  IB 

0.0012 

2.58 

Log  survival  time 

0.53 

0.0297 

0.60 

0.0120 

2.48 

Log  survival  time 

0.79 

0.0188 

0.69 

0.0060 

3.13 

These  somewhat  artificial  but  valid  comparisons  are  compatible  with 
the  idea  that  precision  of  graded  response  may  be  a  little  more  than  double 
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that  of  a  quantal  response,  and  that  1/b  and  s/bg  tend  to  be  close.  Such 
values  as  0.53  and  0.60,  for  instance  are  df  the  same  order.  The  com¬ 
parisons  are  undertaken  only  to  give  a  rough  check  to  the  theory  of  relation 
of  responses,  and  are  not  advised  as  a  procedure  for  experimenters. 

The  agreement  of  closely  corresponding  responses,  such  as  those  In  the 
table,  does  not  Indicate  agreement  of  all  responses.  In  the  problem  repre¬ 
sented  in  the  third  row  of  the  table,  when“percentage  of  death  a  was  used 
as  an  all-or-none  response,  1/b  was  much  above  s/bg.  In  the  problem 
of  the  fourth  row,  1/b  for  death  as  a  response,  was  lower  than  s/bg.  The 
approximate  agreement  seems  to  occur  when  both  are  about  equally  well 
adopted. 

At  Fort  Detrick  some  study  has  been  given  to  use  of  graded  response 
in  the  hope  of  a  gain  in  precision  over  the  often  difficult  all-or-none  testa. 
Responses  studied  have  Included  time  to  denth,  time  to  onset  of  symptoms 
and  weight  loss.  In  some  cases  the  graded  responses  have  shown  some  ' 
gain  over  quantal  responses;  in  others  they  have  given  difficulty 'and  have 
failed  to  compete  with  the  all-or-none. 

Since  for  equal  precision  l/b2w  -  s2/b2  ,  and  since  w  may  be  about 

2  y  o  2 

0.5  or  a  little  less,  2/b  may  bo  equated  to  i  A  ?.  Solution  of  this 
equation  should  give  the  required  for  approximate  equivalence  to  a  given 
s/bg  or  vice  versa.  For  example,  in  a  recent  test  s/bg  was  estimated  at 
0.  76.  Writing  2/b2  =  s2/bZg  (or  0.58),  and  solving,  a  value  of  b  ■  1.86 

Is  indicated  as  competitive  In  precision.  PerhapB  2.5  would  be  a  better 
factor  than  2  for  this  comparison  since  0,4  is  probably  nearer  the  usual 
average  weight  than  0.5.  This  procedure  has  been  helpful  In- the  writer** 
work,  and  should  prove  of  value  to  expert  mentors. 

In  making  a  choice  between  responses,  the  first  criterion  will  be  the 
adaptation  of  available  responses.  The  equation  just  above  may  be  of  help. 

If  graded  and  all-or-none  response  are  equally  well  adapted,  the  graded 
response  may  be  expected  at  least  to  double  the  precision  of  the  all-or-none. 
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.  A  COMPARISON  OF  LABORATORY  EVALUATION  AND 
FIELD  WEAR  OF  MILITARY  FABRICS 

William  S.  Cowl* 

Quartermaster  Research  and  Engineering  Command 
Natick,  Massachusetts 

The  Textile,  Clothing  5  Footwear  Division  of  the  Quartermaster  Research 
&  Engineering  Command  is  charged  with  the  responsibility  for  development 
of  new  and  improved  textile  fabrics  for  military  garments  and  other  textile 
end  Items  such  as  tentage  and  personal  equipage  for  metahers  of  the  ormed 
forces.  This  paper  deals  with  certain  problems  which  have  arisen  In  the 
evaluation  of  new  textiles  for  clothing  Items. 

The  normal  pattern  of  development  far  e  new  textile  fabric  for  military 
use  is  es  follows: 

1.  ENGINEERING  OF  THE  FABRIC.  Engineering  design  of  new  textile  fabrics 
is  accomplished  by  textile  technologists  located  In  the  Textile,  Clothing  & 
Footwear  Division  at  Natick,  Mass.  Dependent  upon  the  functional  character¬ 
istics  desired  in  the  end  item  for  whit*  the  fabric  ts  Intended,  appropriate, 
weights,  weaves,  yam  sizes,  textures  end  finishes  are  decided  upon  and 
tentative  specifications  are  prepared.  In  many  instances  the  feasibility  of 
fabrication  of  an  experimental  fabric  is  discussed  with  the  representatives 

of  the  textile  Industry  before  these  tentative  specifications  are  finalized.  ' 

2 .  PROCUREMENT  OF  SAMPLE  YARDAGE.  The  textile  Industry  is  Invited  to 
bid  on  contracts  for  the  fabrication  of  sample  yardages  of  experimental  fabrics. 
Such  contracts  usually  call  for  the  production  of  from  500  to  1000  yards  of 

the  new  fabric.  It  is  customary  for  one  of  the  textile  technologists  Involved 
In  each  new  textile  designate  visit  the  contractor's  plant  In  order  to  observe 
the  fabrication  process  and  to  discuss  with  the  contractor  any  difficulties 
which  may  have  arisen  during  manufacturing  operations. 

3.  LABORATORY  EVALUATION  OF  THE  FABRIC.  When  fabrication  of  the  ex¬ 
perimental  item  has  been  completed,  the  cloth  Is  shipped  to  the  Textile 
Engineering  Laboratory  at  Natick  in  order  that  a  complete  physical  evalu¬ 
ation  may  be  accomplished.  The  laboratory  fs  a  completely  equipped  tex¬ 
tile  testing  facility  containing  all  of  the  test  instruments  prescribed  by  the 
American  Society  for  Testing  Materials  as  well  as  research  equipment  as 
required. 

New  textile  fabrics  are  checked  for  both  constructional  and  physical 
requirements  contained  in  the  tentative  specification.  Amongst  the  former 
are  weave,  weight,  yarn  size,  yam  count,  texture  (the  number  of  warp  and 
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filling  yams  per  inch),  thickness,  type  of  fiber  or  fibers  etc.  Some  of  the 
important  physical  requirements  are  breaking  strength,  tearing  strength, 
bursting  strength,  resistance  to  abrasion,  porosity,  elongation,  water  re- 
pellency  find  others.  Of  course  the  relative  importance  of  these  physical 
characteristics  is  dependent  upon  the  end  iter*  use  for  which  the  new  fabrics 
are  candidates.  In  every  instance  new  items  ace  compered  to  the  existing 
fabric  which  they  may  replace.  For  those  characteristics  which  may  be  ex¬ 
pressed  in  terms  of  numerical  data  {such  as  breaking  strength  for  example), 
differences  between  the  standard  and  experimental  item  are  tested  statisti¬ 
cally  by  means  of  such  standard  techniques  as  the  *t“  and  F  teats,  and  the 
analysis  of  variance. 

4.  FIELD  EVAIUAT-OM.  Those  experimental  fabrics  which  show  promise  on 
the  basis  of  laboratory  evaluation  are  fabricated  into  garments  (specifically 
trousers)  at  the  clothing  factory  of  Philadelphia  Quartermaster  Depot.  These 
trousers  ere  forwarded  to  the  Field  Eva  luatlon  Agency  of  the  Quartermaster 
Research  &  Engineering  Command  which  is  located  at  Ft.  Lea,  Va.  Here  they 
are  subjected  to  accelerated  field  wear  on  a  specially  designed  fabric  wear 
course  (Figure  1  is  placed  at  the  end  of  this  article).  Standard  trousers  for 
comparison  are  worn  over  the  course  simultaneously.  All  garmertta  are  worn 
by  military  test  subjects. 

*  ^  A  * 

The 'fabric  course  Is  a  quarter  of  a  mils  long  and  consists  of  thirty  ob¬ 
stacles.  The  test  subjects  climb  a  stone  embankment,  crawl  across  a 
section  of  railroad  track,  and  slide  down  a  steep  cobblestone  incline.  They 
crawl  across  a  single  log  bridge,  through  concrete  culverts,  across  terrain 
consisting  of  cinders,  sand,  gravel  and  boulder*.  They  also  crawl  through 
trenches  and  across  rough  terrain. 

Two  traversals  of  the  course  constitute  one  cycle.  After  each  cycle 
garments  are  laundered  and  a  wear  score  Is  obtained  based  on  visual  exami¬ 
nations  by  trained  military  personnel  who  chart  the  scores.  Frays,  holes, 
tears  and  wear  areas  are  all  considered  in  computation  of  the  total  wear 
ucore.  Depending  upon  the  severity  of  each  of  these  types  of  wear,  a  point 
value  Is  assessed.  At  the  completion  of  ten  cycles,  wear  scores  for  both 
the  experimental  and  standard  garments  are  totalled.  The  results  are  com¬ 
pared  statistically  by  the  analysis  of  variance  technique  and  a  formal  written 
report  is  prepared  and  submitted  to  Headquarters  Quartermaster  Research  & 
Engineering  Command. 
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5.  USER  TEST.  If  the  results  of  field  evaluation  of  an  experimental  fabric 
indicate  significant  improvement  over  the  standard  fabric,  then  consideration 
is  given  to  fabrication  of  a  large  number  of  garments  containing  the  new  fabric 
If  the  excision  is  made  to  do  so,  these  garments  are  pieced  In  the  hands  of 
troops  located  at  various  military  installations .  The  choice  of  location  de¬ 
pends  upon  the  specific  garment  for  which  the  new  fabric  is  intended.  For  ' 
example,  faMcs  for  col’d  weather  clothing  might  be  tested  in  Alaskan  babes, 
and  garments  containing  lightweight  tropical  fabrics  might  be  worn  In  Panama. 
A  report  is  prepared  of  the  user's  reaction  to  the  new  fabric. 

5.  STAlT0*Ppi.7.*?Tp'T.  When  the  user’s  reaction  to  new  fabrics  is  favorable, 
action  is  taken  to  finalise  the. temporary  requirements  for  the  fabric  which 
have  berm  in  effect  up  to  this  time.  A  final  formalized  specification  U  pre¬ 
pared  over  which  large  quantities  of  the  fabric  may  be  procured  end  the  new 
fabric  replaces  the  present  standard.  In  the  event  that  this  new  fabric  is  in¬ 
tended  for  an  end  item  which  Is  of  interest  to  services  other  then  the  Army 
then  the  new  specification  is  coordinated  with  these  other  services  prior  to  . 
promulgation. 

It  is  realized  thdt  she  above  background  material  is  very  much  of  an 
oversimplification,  however,  the  information  is  provided  only  to  introduce 
the  specific  problem  with  which  this  paper  is  concerned. 

As  was  noted  earlier,  only  those  new  fabrics  which  show  promise  in  the 
laboratory  are  evaluated  in  the  field.  Historically,  good  correlation  has  been 
obtained  between  the  results  of  laboratory  flex  abrasion  testing  and  wear 
scores  obtained  on  the  fabric  wear  course.  Reasonably  good  correlation  has 
also  been  obtained  between  tear  resistance  as  determined  in  the  laboratory 
and  field  evaluation  results  (Ref.  1).  Finally,  good  correlation  has  been 
found  between  accelerated  field  wear  cjn  the  fabric  course  and  actual  field 
wear  on  infantry  training  troops.  Over  the  period  of  years  between  1945  to 
1958  the  presence  of  good  laboratory  -  field  correlation  was  verified  on 
numerous  occasions.  However,  all  ol  the  studies  conducted  during  this 
period  dealt  with  all-cotton  garments. 

In  recent  years,  due  to  the  increased  demands  for  durability  and  protec¬ 
tion  imposed  by  modem  warfare  concepts,  interest  in  blends  of  cotton  with 
'synthetic  fibers  has  increased,  ft  is  considered  from  the  known  physical 
properties  of  nylon  for  example,  that  a  more  durable  utility  garment  could  be 
developed  from  proper  blending  of  nylon  with  cotton.  As  a  result,  a  fabric 
was  recently  engineered  from  a  cotton/nylcn  blend  (approximately  70%  cotton 
and  30%  nylon).  When  tested  in  the  *.ab»wa.' lory  samples  cf  trie  new  fabric 
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showed  from  twice  to  six  times  as  much  resistance  to  flex  abrasion  on  con¬ 
ventional  laboratory  equipment  as  did  the  all-cotton  fabric  which  is  currently 
used  in  the  standard  utility  garment.  Such  differences  based  on  past  exper¬ 
iences  would  indicate  that  markedly  superior  resistance  to  wear  would  be 
demonstrated  on  the  wear  course.  However,  when  the  two-fabrics  were 
manufactured  lntcj  garments  and  worn  on  the  wear  course  in  a  two  phase 
evaluation  r.o  significant  difference  was  found  between  the  wear  scores  of 
the  two  items.  In  fact  In  one  phase  of  the  test  the  all-cotton  fabric  appeared 
to  be  slightly  more  resistant  to  wear  than  did  the  cotton/nylon  blend. 

Both  laboratory  and  fabric  course  test  results  were  carefully  re-evaluated. 
No  testing  afterfacts  were  uncovered  which  could  in  any  way  account  for 
the  Undings.  As  a  result  the  following  actions  were  taken. 

A  senior  Textile  Technologist  from  the  Natick  laboratories  visited  Ft. 

Lee  and  pcrror;;»ily  "ran"  the  Combat  Course.  Following  this  experience” 
the  t-jchnolcjist  returned  to  Natick  and  designed  e  new  type  of  laboratory 
abrading  instrument  which  In  his  opinion  more  nearly  reproduced  the  type 
of  wear  encountered  on  tho  fabric  course  than  did  existing  laboratory  test 
equipment.  Th*  Sand  Abiader  is  composed  of  a  block  of  iron  measuring  at 
the  bearing,  surface  2"  x  3M  to  which  a  1/2  Inch  thick  wool  felt  is  cemented. 
The  fabric  to  be  tested  is  clamped  or  sewed  over  the  felt  covered  surface. 

By  means  of  a  pivoted  arm  ihe  block  of  Iron  with  fabric  attached  is  pushed 
back  and  forth  over  a  comeht  block  at  the  rate  of  88  strokes  per  minute. 

Sand  that  has  passed  through  a  4>16  screen  is  constantly  being  dropped  onto 
the  cement  block.  The  sand  Is  sifted  through  a  #30  screen  before  being 
used  again.  The  pressure  on  the  fabric  used  by  the  weight  of  the  arm  and 
iron  block  is  0.5  pound  per  square  inch  which  is  the  pressure  of  e  man'« 
thigh  when  lying  prone.  (Most  of  the  wear  on  the  trousers  in  the  Fabric 
Evaluation  Course  is  on  the  front  of  the  trousers  between  the  knees  end 
crotch). 

Fifteen  samples  of  the  two  fabrics  Included  in  the  above  fabric  course 
wear  studies  have  been  abraded  for  3000  cycles  on  this  instrument.  Visual 
examination  cf  the  abraded  samples  by  a  panel  of  three  textile  technologist* 
revealed  no  less  wear  on  the  cotton/nylon  items  than  on  the  all-cotton 
standard.  Additionally,  tear  strength  values  were  obtained  on  new  sample* 
of  both  materials  and  on  the  abraded  items.  Losses  in  tear  strength  fol¬ 
lowing  abrasion  wero  almost  identical  in  the  filling  direction  of  both  fabric* 
and  only  slightly  greater  in  the  warp  direction  of  the  all-cotton  fabric.  Thus* 
these  preliminary  results  show  much  better  agreement  with  the  fabric  course 
than  did  i'ne  sesults  of  conventional  laboratory  testing. 
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These  results  ore  considered  encouraging  Insofar  as  providing  a  pos¬ 
sible  means  of  improved  correlation  between  laboratory  and  accelerated 
field  wear.  Samples  of  other  new  blends  which  are  scheduled  for  fabric 
course  evaluation  in  ihe  near  future  are  being  similarly  tested. 

However,  it  is  considered  by  the  textile  engineering  staff  of  the  QM  R  • 
and  F.  that  the  solution  to  f -'vs  problem  may  not  lie  in  the  area  of  fabric 
course  -  laboratory  correlation.  No  knowledge  exists  of  the  behavior  of 
similar  blended  fabrics  in  actual  field  wear.  It  may  be  that  the  previous 
correlation  between  accelerated  wear  on  the  fabric  course  and  aatual  field 
wear  which  existed  on  all-cotton  fabrics  may  not  be  found  when  part  syn^'.  V 
thetlc  garments  aro  tested.  Therefore,  in  addition  to  the  development,  a 
small  scale  pilot  study  is  presently  underway  on  combat  troops.  Twelve 
members  of  an  artillery  battalion  are  wearing  utility  ensembles  fabricated 
from  a  cotton/nylon  blend  in  maneuvers  and  during  training.  These  artil¬ 
lerymen  have  also  been  Issued  new  all-cotton  uniforms  for  comparison.  ■■ 
Admittedly,  this  is  not  a  controlled  or  designed  experiment.  Neither  fabrio 
or  resources  ere’&vailabie  for  a  more  ambitious  effort  at  this  time.  It  la 
anticipated,  however,  that  valuable  information  may  be  obtained  which  will  . 
assist  in  the  design  of  a  formal  field  trial  which  is  planned  for  the  spring 
of  1961.  At  that  time  a  large  scale  field  wear  evaluation  will  be  conducted 
at  Ft.  Jackson,  S.  C.  on  two  full  platoons  of  soldiers  wearing  experimental 
vs.  standard  utility  uniforms.  Although  this  design  Is  not  complete  et  this 
time,  it  is  hoped  that  appropriate  statistical  techniques  will  help  enable  the 
Quartermaster  Corps  to  glean  the  greatest  amount  of 'possible  information  ■ 
from  this  study  end  will  also  provide  means  for  assessing  new  and  promising 
fabric  developments  for  military  garments  with  the  least  expense  and  shortest 
lead  time. 
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GROUP  SCREENING  DESIGNS 

W.  S.  Connor 
Research  Triangle  Institute 

1*  INTRODUCTION.  Recently,  G.  8.  Watson  [lj  considered  a  partic¬ 
ular  approach  to  the  problem  of  screening  a  large  number  of  factors  of  which 
only  a  few  affect  the  response  variable.  The  general  approach  la  similar  to 
that  of  Dorfman  f  23  to  the  biological  problem  of  the  detection  of  a  rare  de¬ 
fect  a  mono  the  members  of  a  large  population..  Dorfman  suggests  that  pool¬ 
ed  blood  samples  be  tested,  end  that  the  individual  samples  which  form  a 
pooled  sample  be  tested  whenever  the  latter  gives  a  positive  result.  A  100  ■ 
percent  screening  may.  be  achieved  with  substantial  saving  in  the  number  of 

blood  tests.  The  present  paper  modifies  the  development  in  TO  *o  that 
orthogonal  designs  may  be  used. 

2.  THE  GROUP  SCREENING  DESIGN  WHEN-  THERE  IS  NO  7TXPSRI  MENTAL 
ERROR.  Baginning  with  the  case  when  the  experimental  error  is  negligible, 
Watson  makes  the  following  assumptions: 

(I)  all  factors  have,  independently,  the  same  prior  probability,  p 
(q  -  1  -  p),  of  being  effective, 

(II)  a  factor  is  effective  if  it  produces  a  non-zero  change  In  the  rei- 
sponse, 

(III)  none  of  the  factors  interact, 

(lv)  the  directions  of  possible  effects  are  known,  and 

(v)  the  number  of  factors  f  *  gk,  where  g  “  tho  number  of  groups  and 
k  ■  the  number  per  group. 

A  typical  group  screening  design  is  illustrated  by  f  ■  9.  Before  discus¬ 
sing  It,  we  note  that  for  a  single  stage  design,  In  the  absence  of  experi¬ 
mental  error,  ten  runs  ers  sufficient  to  determine  which  factors  are  effective 

i 

^  •  » 

- - . 
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Of  course,  this  design  will  not  permit  ungorrelated  estimates  of  the  main  ef¬ 
fects.  In  the  group  screening  design,  the  nine  factors  denoted  by  A,  B,  . .., 
I,  are  divided  into  g  *  3  groups  of  It  ■  3  factors  each,  to  form  group-fac¬ 
tors  (A,  B,  C),  (D,  E,  F),  ana  (Q,  H,  I),  which  may  be  denoted  respectively 
by  X,  Y,  and  Z,  Then,  adoptinp  the  convention  that  the  upper  and  lower  lev-' 
els  of  the  factors  A,  B,  ...,  I  are  ta.Un*d  so  that  their  effects.  If  any,  are 
to  give  greater  and  lesser  valuvi,  to  the  response,  so  that  the  main  effects, 
if  any,  are  positive,  the  upc-v  and  loviter  levels  of  the  group-factors  nre  de?- 
fined  os  follows: 


Definitions,  of  l  evels  of  Grpifn -Factors  1 
Group-Factors 

(A,B,C)dC  (D,E,F):Y  <G,H,I)jZ 

Lower  level  (0,0, 0)d  (0.0#0)d 

Upper  level  (1,1,  l)tx  fl,l,l):y  • 


The  first  stage  design  Is  for  the  group -factors, 
plicate  of  a  23  design,  as 


(0,0„0):1 

U,l,l):s 

One  may  use  a  1/2  re - 


(2.1) 

Levels 


(2.2) 


x,  y#  t,  xy*. 


In  terms  of  the  factors,  these  treatment  combinations  are 

(2.3)  x:(l,  1,1,  0,  0,  0,0,  0,0) 

y:  (0,0,0,  1,1, 1,  0,0,0) 

*:{0,  0,0,  0,0,  0,1, 1,1) 

xyx:  (1, 1, 1, 1,  1, 1. 1, 1,  l) 


The  usual  functions  of  the  responses  to  the  treatment  combinations  era 
indicated  below: 
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Combine  tlon 

Maan. 

Main  Effect 

I  1  1 

X 

1 

1 

-1  -1 

r 

1 

-1 

1  -1 

£ 

1 

-I  . 

-1  l 

xyz 

l 

1 

1  1 

v 

\ 


For  our  purposes,  the  divisor  will  be  taken  as  the  number  of  responses 
In  the  design,  which  in  thU  Instance  Is  four.  In  view  of  (Hi),  this  design 
will  estimate  the  main  effects  of  the  group-factors,  and  In  view  of  (Hi)  and 
(lv),  there  will  be  no  cancelling  out  of  effects  within  the  group-factors. 


Every  group-factor  which  contains  at  least  one  effective  factor  will  It¬ 
self  be  effective  end,  because  of  the  absence  of  experimental  error,  will  be 
detected.  If  a  fiTSt  stage  experiment  reveals  that  one  or  more  group-factors 
are  affective,  a  second  stage  experiment  will  be  carried  out  on  the  factors 
which  comprise  them  to. find  out  which  factors  are  effective.  For  example, 

U  group-fectorX  is  effective,  but  group-factors  Y  and  Z  are  not  effective, 
than  the  second  stage  experiment  will  involve  factors  A,  B,  and  C.  If 
u  cup -factors  X  and  Y  ore  effective,  but  not  group -factor  Z,  then  the 
second  stage  experiment  will  involve  factors  A,  B,  C,  D,  E,  and  F. 
Finally,  If  group-factors  X,  Y,  end  Z  are  effective,  then  all  nine  factors 
will  be  included  in  the  second  stage  experiment.  Accordingly,  depending 
on  the  outcome  of  the  first  stage  experiment,  there  may  be  no  second  stage 
at  all,  or  there  may  be  a  second  stage  involving  3,  6,  or  9  factors. 


If  h  factors  are  to  be  studied  at  the  second  stage,  then  only  h  runs 
are  needed  at  the  second  stage.  This  Is  because  one  run  from  the  first 
stage  can  be  used  In  the  analysis  of  the  responses  from  the  second  stage. 
To  illustrate,  suppose  that  only  group-factor  X  is  effective  from  the  first 
stage.  Then  factors  A,  B,  and  C  must  be  studied  In  the  second  stage. 
One  way  that  this  can  be  done  Is  by  running  treatment  combination! 


♦Actually,  it  oan  be  demonstrated  that  If  n  group-factors  are  effective, 
then  only  n(k-l)  runs  ere  needed  at  the  second  stage,  not  nk  -  h  as  stated 
here. 


at  the  second  stage.  Then,  remembering  that  x  -  abc, . the  effects  of  A.  B, 
and  C  are  determined  from 


(2.6)  ' 


-  be,  x  -  ec,  and  x  -  ab. 
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be  -  «D,  1,1.  0,0,  0,0,0.  0) 
ac-  a,  0, 1,  0.0,  0,0,  0,0) 
ab  -  (1,1.  0,  0,  0,  0,  0,  0,  0) 


•  '.v.'-v-' 

iNV.V.V.  .- 
i  v  v  ■„« 


respectively. 

By  assuming  some  Value  for  p,  it  is  possible  to  calculate  the 
ties  of  various  numbers  of  runs  at  the  second  stage,  and  thereby  to  compare 
the  group  screening  design  with  the  single  stage  design.  For  p  -  .15, 
fp- 1.35,  so  that  it  is  expected  that  one  or  two  of  the  nine  factors  will  be 
effwctlve.  The  probabilities  that  a  group-factor  wiU  contain  0,  l,  *1  or 
3  offcctive  factors  are  given  below: 

12.7)  Probabilities  that  a  Group-Factor  Will  Contain 

or  3  Effective  Factors,  for  “  -  ** 


Number  of 
Effective 


V.~J 


S: 


Numerical 
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*  . 
If  there  are  1,  2,  or  3  effective  factors,  the  three  factors  will  be  run 
In  a  second  stage.  The  probability  of  this  event  is 

(2.8)  1  -  qZ  *  1  -  .614  •  .386  «  r,  say. 


There  -will  be  no  second  stage  if.  there  are  no  effective  factors.  The 
probability  of  this  event  is  q9.  The  probabilities  that  the  aecond  stage  will 
require  0,  3,  6,  or  9  runs,,  end  therefore  that  the  two  stages  together 
will  require  4,  7,  10,  or  13  runs  are  given  below: 


(2.9) 


Probabilities  of  Various  Numbers  of  Runs 


Number  of  Runs 
Second 


Numerical 


Stace 

total 

Formula 

Value 

'  0 

* 

4  .  •  •  • 

(1-r)3 

.23 

4 

3 

7 

3r(l-r)Z  ’ 

.44 

6 

10 

3rZ(l-r.) 

.27 

9 

13 

V 

.06 

From  this  table. 

the  expected  total  number  of  runs  is  calculated  to  be 

(2.10) 

4  x  0.23  +  7  x  0.44  • 

f  10  x  0.27  +13  *0. 

06  -  7.48. 

which  la  an  average  saving  of  2.52  runs  from  the  10  runs  which  are  required 
by  a  single  stage  experiment. 

The  saving  is  greater  for  smaller  p.  For  p  -  .10.  the  expected  number 
of  runs  is  6.43  and  for  p  *  .05,  it  is  5.29,  Of  course,  the  number  of  runs 
cannot  drop  below  4.  For  p  greater  than  .15,  the  saving  is  less.  For 
p  »  .20,  the  expected  number  of  runs  Is  8.39  and  for  p  -  ;29,  the  expected 
number  of  runs  is  '  10,  so  that  for  still  larger  p,  the  single  stage  design  is 
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preferable.  These  observations  illustrate  the  general  principle  that,  for 
fixed  f,  the  expected  saving  varies  Inversely  with  p. 

3.  THE  GROUP  SCREENING  DESIGN  WHEN  THERE  13  EXPERIMENTAL 
ERROR.  For  the  case  when  the  experimental  error  is  appreciable,  o'  t  0, 
Watson  modified  assumption  (11)  to 


(11)  effective  factors  have  the  same  effect,  A  >  0. 


This  assumption  implies  that  the  effect  of  a  group-factor  is  one  of  the  val¬ 
ues  0,  A,  2  A  kA  and  that  If  the  effect  is  sA,  (s»0,,..., 

k),  then  the  group-factor  contains  s  effective  factors  and  '(k  -  s)  Ineffec¬ 
tive  factors.  But,  of  course,  in  the  real  problem,  the  effect  of  a  group- 
factor  may  be  some  value  other  than  A  ,  and  affects  s  A  may  be  achieved 
by  adding  effects  from  s'  f  i  factors.  Although  this  assumption  is  some¬ 
what  arbitrary  and  unrealistic,  It  perhaps  results  in  shedding  some  light  on 
the  characteristics  of  group  screening  designs.  It  is  akin  to  the  real  problem 
in  that  the  levels  of  the  factors  may  be  chosen  in  such  a  way  that  there  Is 
a  common  least  change  in  response,  say  At,  which  Is  worth  detecting. 

Another  assumption  made  by  Watson  is  that 

(vl)  the  errors  of  all  observations  are  Independently  normal  with  a  con¬ 
stant  known  variance  . 

The  procedure  Is  further  specified  by  assuming  that 

(vll)  estimated  main  effects  of  group-factors  are  tested  at  significance 
level  ot,  and  If  one  or  more  of  them  is  significantly  different 
from  zero,  a  second-stage  experiment  Is  carried  out.  Tests  of 
whether  the  main  effects  of  the  factors  are  zero  are  made  at  sig¬ 
nificance  level  £• 

*  Because  of  the  nice  properties  of  orthogonal  designs,  it  will  be  assumed 
that  such  designs  are  used  at  both  stages.  Orthogonal  fractional  factorial 
designs  exist  having  2m  treatment  combinations,  which  can  be  used  to 
estimate  the  main  effects  of  2m  -  l  factors.  This  is  a  rather  thin  series. 
However,  Plackett  and  Burman  lO  give  orthogonal  designs  having  4t 
treatment  combinations,  which  will  accommodate  4t  - 1  factors. 
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For  the  example  under  discussion,  suppose  that  the  first  Btage  design 
Is  the  one  already  described  above.  If  there  Is  only  one  group-factor  which 
is  found  to  be  significant,  then  three  factors  are  varied  In  the  second  ex¬ 
periment,  end  tho  same  design  can  be  used.  If  two  group-factors  are  fotind 
to  bo  significant,  then  eight  treatment  combinations  are  needed  at  the  second 
stage.  Such  a  design  Is  given  below: 


* 


(3.13  Treatment  Combinations  iu  Six  Factors 

A  fi  .S  S  E  F 

o  o  o  o  o  o  i  o 

0  0  0  1  11*1 

0  1  10  0  1(1- 

0  1  1  110  I  o 

1  0  l  0  1  o  I  1 

I  0  l  I  0  1.  0 

i  i  o  o  i  i  ;  o 

II  0  10  0  1  1 


A  seventh  factor  could  be  added  at  the  levels  shown  in  the  last  column,  or 
the  three  remaining  factors  all  could  be  held  constant  throughout. 


If  all  three  group-factors  turn  out  to  be  significant,  then  twelve  treet- 
ment  combinations  are  required  at  the  second  stage.  A  suitable  design  Is  the 
following: 

(3.2)  Twelve  Treatment  Combinations  for  Nine  Factor* 

_ (Plackett  and  Burman)  _ 


A  JL 
i  o 
i  l 
0  1 
1  0 
1  1 
1  1 
0  1 
0  0 
0  0 
1  0 
0  1 
«  0 


SL  J& 
1  0 
0  1 
1  0 
1  1 
0  1 
1  0 
1  1 
1  1 
0  1 
0  0 
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A  design  accommodating  two  mere  factor?  cen  be  obtained  by  assigning  levels 
as  indicated  in  the  last  two  columns. 

The  expected  number  of  runs,  the  expected  number  of  effective  factors 
detected,  and  the  expected  number  of  ineffective  factors  wrongly  declared 
to  be  effective  are  quantities  which  describe  the  operating  characteristics  of 
the  method. 

The  pov>er  of  the  test  of  a  group -factor  depends  on  its  mean.  If  the  mean 
Is-  s'.  A  ,  the  power  of  the  t-test  of  it  will  be 

(3.3)  Tf  X  »  1'1  (S  cj>i,  a.), 

where  • 

4>i  -  /»  [il 

Is  the  parameter  used  In  Table  to  of  Pearson  and  Hartley  Then  the 

probability  that  a’ group-factor  will  be  declared  significant  la 


(3.4)  *  ■  Jk  . 

ri  ■  L  (Slp*qk“  yr<,<Pi‘  «>• 

s  “  0 


and  that  an  effective  group-factor  will  be  declared  significant  is 

ir,'  -[£  fl)p*  qk"  <f>j.  col/a-ib. 

1 


and,  of  course,  the  probability  that  an  ineffective  group-factor  will  be  de¬ 
clared  significant  is  at. 
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The  power  of  the  t-test  of  a  factor  will  be 

(3.5)  Tf  j  *  T2  (  4> 
where 

ft  2  m  J2  [J^-]r  ' 

“7  »  0  for  an  ineffective  factor  and  1  for  an  effective  factor,  and 

is  the  least  Integer  greater  than  -0^-  ,  except  that  [•&&-]  ■«  0  when  n«0. 

Of  course,  IT 2(0,  j3  ) 

It  can  he  shown  that  the  expected  number  of  effective -factors  to  be  de¬ 
clared  effective  {significant}  is 

(3.6)  E  -  kp-  £  „  TCj  (  h  [  4-J  -A.,^kjd r,*"  o-»*)  «•** 

n*0  ' 

where  p’  «  p  Tf  ^'/  ,  end  that  the  expected  number  of  ineffective 

factors  to  be  declared  effective  is 

•  l  .. 

(3.7) ‘  E  “  fq*  [tf*  -  q^Mr/  -  tfp)]  /0~qk). 

Also,  the  expected  number  of  runs  is 

*  ■  4 1  *H  +  Z  *  MMfll)  r,*  "a  - 

n-1 


(3.8) 
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These  formulas  may  be  calculated  for  the  example  under  consideration. 
They  are  as  follows:. 


2  A/rr 


V 

K*  -0.614  a  +0.325  +  0.057  >^(2  .^Ot)  1 

+  0.003  iri (3  4>1.  OL) 

\  ■  ■  .  ■ 

0.644  7Tx((pv  a)  +  0.148  Dc)  +  0.008  ^,(3^.  (X.) 

3  IL  <o.ls>  £  o-n*)3'” 


r: 


E  -  lS.l^fc* -0.723  (EXO.IS))  -  19.1^(0.2777^!**  O.lOOOLj 


«.4i£  i[f](S)  v,'“  o-r;>5-". 


n“l 


From  (2,7)  It  is  seen  that  for  p  -  .15,  the  probability  of  two  or  more  ef¬ 
fective  factors  occurring  together  in  the  same  group-factor  is  equal  to  .06. 
Accordingly,  in  practice,  one  would  not  have  to  know  the  directions  of  pos¬ 
sible  effects,  and  two-sided  tests  would  be  used.  Some  calculations  for  E, 

E  and  R  have  been  made  for  two-sided  tests.  For  A/or  ■  12,  2,  3; 

0.01,  0.05;  and  £  -  0.01,0.05,  the  results  are  shown  In  the  accompany 

ing  table. 
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As  might  have  been  anticipated,  Increasing  a  or  A/cr  results  In  mar* 
group-factora  being  declared  significant  and,  hence,  results  in  more  factor* 
being  tested  in  the  second  stage.  It  therefore  increases  the  average  number 
of  rune,  the  average  number  of  effective  factors  which  are  identified,  and  • 
less  markedly,  the  number  of  ineffective  factors  which  are  declared  to  bo 
effective.  Increasing  J  has  no  effect  on  the  average  number  of  runs,  but 
does  increase  the  average  number  of  effective  factors  which,  are  identified 
and  the  average  number  of  ineffective  factors  which  are  declared  to  be 
effective. 

REFERENCES 

M  Watson,  G.  S.,  "A  study  of  group  screening  designs,  *  Technical  Re- 
1  s  port  No.  2,  S-10,  Statistics  Research  Division.  Research  Triangle  In¬ 
stitute;  to  be  published  in  Technometrlc_f . 

k]  Dorfman,  R.  .  "The  detection  of  d- 'active  members  of  large  populations,' 
Annals  of  Mathematical  Statistics,  Vol.  14  (1943),  pp.  436  -  440. 

(jl  Pearson,  E.  5.  and  Hartley,  H.  O. ,  Bromatrika  Tables,  fgjr  Statistician*, 
‘  J  Vol.  1,  Cambridge  University  Press,  1954. 

U]  Plackett,  R.  L.  and  Burman,  J.  P.,  "Design  of  optimum  mUlti -factorial 
1  1  experiments."  Blcmetrika^ Vol.  33.  1946.  ... 


MULTIVARIATE  ANALYSIS  ILLUSTRATED  B!f  NIKE  -  HERCULES: 

I. .  Separation  of  product  and  measurement  variability.  \ 

H.  Acceptance  sampling. 

I.  Edward  Jackson 
.  ,  Eastman  Kodak  Company 

Abstract 

This  expository  paper  is  concerned  with  the  application  of  some  multi¬ 
variate  techniques  to  some  of  the  problems  involved  in  evaluation  of  miaslla 
testing  data.  Examples  Illustrating  these  techniques  are  drawn  from  actual 
Nike  booster  test  data. 

The  first  pert  of  the  paper  is  concerned  with  methods  separating  the  total 
variability  of  teat  results  into  components  representing  product  and  measure¬ 
ment  variability.  Specific,  techniques  discussed  are  (1)  regression  analysis, 
(2)  principal  componnnte,  and  <3)  the  methods  of  Grubbs,  Kruakal  and  David 
for  analyzing  related  pairs  of  observations. 

The  problems  of  acceptance  sampling  when  more  than  one  variable  la  In¬ 
volved  are  discussed  in  the  second  part  of  the  paper.  Because  of  the  cost 
of  testing  involved  In  missile  evaluation,  a  sequential  multivariate  type  of 
inspection  plan  is  developed.  Considerable  emphasis  Is  placed  on  the  pro¬ 
blems  Involved  in  incorporating  the  product  specifications  Into  these  sam¬ 
pling  plans.  A  second  procedure  is  developed  to  sequentially  test  for  excess 
dispersion  within  a  lot. 


MULTIVARIATE  ANALYSIS  ILLUSTRATED  ffif  NILE -HERCULES 

J.  Edward  JackBon  * 

Eastman  Kodak  Company,  Rochester,.  New  York 

I.  SEPARATION  OF  PRODUCT  FROM  TESTING  AND  MEASUREMENT  VARIABILITY 

1.  Introduction.  In  the  past  few  years,  the  techniques  associated  with 
multivariate  analysis  have  begun  to  come  into  their  own  in  the  field  of  in¬ 
dustrial  statistics.  Prior  to  that  time,  say  World  War  H,  most  of  the  avail¬ 
able  literature  was  devoted  to  the  use  of  factor,  analysis  in  education  and 
psychology  or  the  use  of  discriminant  analysis  in  genetics  and  archeology. 
However,  by  the  close  of  World  War  II,  we  in  the  industrial  world  began  to 
discover  that  control  charts  and  the  analysis  of  variance  would  not  solve  ell 
of  our  problems  and  that  one  of  the  reasons  for  this  was  that  several  factors 
In  a  production  problem  seemed  to  vary  all  at  once,  but  not  bn  a  completely 
random  fashion.  Starting  with  Hotelling's  bombsight .  paper,'4  the  number' 
of  references  In  the  literature  has  grown  slowly  but  steadily  and  eventually 
the  generalized  T2 -statistic,  multivariate  analysis  of  variance  and  the 
method  of  principal  components  should  become  regular  members  of  the  kit 
of  tools  of  the  industrial  statistician.  There  still  is  a  great  need  for  aome 
fairly  non -technical  literature  on  multivariate  methods  and  more  work  in 
methodology  to  "translate"  the  great  amount  of  theoretical  work  now  avail¬ 
able  into  a  form  readily  usable  by  the  practicing  statistician. 

Mast  published  works  so  far  have  been  rotated  either  to  control  chert 
procedures  or  component  and  factor  analysis.  To  show  that  there  are  other 
Industrial  applications  of  multivariate  techniques,  the  first  part  of  this  paper 
will  deal  with  some  methods  used  to  break  the  total  variability  of  a  system 
into  components  representing  product  variability  end  testing  and  measure¬ 
ment  variability.  Soma  of  the  techniques  used  ere  not  classical  multivariate 
techniques  as  such  but  all  of  them  are  nevertheless  multivariate  in  the  aenae 
that  they  all  take  into  account  the  relationships  among  two  or  more  variables. 
These  techniques  will  be  illustrated  with  numerical  examples  dealing  with 
static  tests  of  Nike  boosters  carried  on  by  the  Hercules  Powder  Company  at 
Radford  Arsenal,  Virginia.  It  should  be  pointed  out  that  the  purpose  of  these 
examples  is  to  Illustrate  the  various  techniques  and  does  not  reflect  in  any 
way  on  the  quality  assurance  policies  exercised  by  the  Radford  Arsenal.  ■ 

It  will  be  seen  that  no  one  of  these  techniques  can  completely  separate 
product  variability  from  testing  and  measurement  variability  and  it.  would 
seem  that  an  Integrated  system  employing  perhaps  severed  of  these  techniques 
would  be  necessary  to  obtain  optimum  result*. 
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2.  Regression  Analysis.  While  not  everyone  may  construe  the  term  "regres¬ 
sion  analysis"  to  be  a  part  of  multivariate  analysis,.  It  nevertheless  should 
be  thought  of  as  such  since  this  technique  tikes  Into  account  the  relation¬ 
ship  of  each  of  the  independent  variables  with  each  other  as  well  as  .with 
tha  dependent  variable.  One  of  the  steps  in  regression  analysis  is  to  ob¬ 
tain  a  sum  of  squares  "due  to  regression"  and  from  this,  break  the- total* 
variability  of  a  system  into  components  representing  explained  and  unex-  . 
plained  variability.  In  experimental  design  problems,  the  factors  which 
one  generally  studies  are  ones  associated  with  tha  product  and  hence  the 
residual  variability  of  an  analysis  of  this  type  of  data  generally  represents 
experimental  error.  In  the  present  case,  the  procedure  will  be  reversed. 

Tire  variables  to  be  considered  will  be  ones  associated  with  the  testing 
and  measurement  phase  of  the  program  so  that  the  major  portion  of  the  re¬ 
sidual  could  be  assumed  to  represent  product  variability.  Hotelling  used 
this  method  to  determine  the  residual  variability  of  bomboights  after  remov¬ 
ing  the  effect  of  guidance  systems,  crews  end  flight  and  bombing  patterns. 

In  the  present  case,  we  shall  be  concerned  with  tha  action  times  of  Nike 
boosters  as  determined  from  static  tests. 

,  #  , 

The  procedure  in  static  testing  consists  of  conditioning  a  round  to  a 
specific  temperature,  then  placing  it  in  a  firing  bay  with  strain  gages 
fastened  to  its  nose  iri  such  a  way  that  pressure,  thrust  and  action  time 
(essentially  a  measure  of  the  time  it  takes  to  bum  ell  of  tha  powder  in  a 
round)  can  be  measured.  The  measurements  arc  recorded  either  on  an  oa- 
cllloucope  or  an  electronic  integrator  using  a  fairly  complex  system  of 
components.  Since  this  is  a  complicated  procedure,  a  number  of  production 
records  are  kept  on  anything  which  might  affect  the  results  and  these  be¬ 
come  the  independent  variables  in  our  regression  analysis. 

The  present  example  will  deal  with  the  variability  of  the  action  time 
measurements  for  a  production  lot  with  the  tested  rounds  being  conditioned 
to  a  temperature  of  -10°lr\  The  independent  variables  are: 

« 

x,  =  Propellant  temperature.  (Although  the  propellant  temperature  is 
supposed  to  be  -10°  F,  it  may  actually  vary  a  degree  or  two  from 
this.) 

x2  »  length  of  time  propellant  was  conditioned. 

x-  -  Outside  tampers  ture.  {This  can  be  as  iaemSMat  lector  for  -10° F 
rounds  if  it  is  90° F  in  the  shade.) 
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x .  ».  Length  of  time  elapsed  from  the  time  the  round  la  removed  from 
its  conditioning  box  until  It  ie  fired. 

x  ■  Length  of  elapsed  time  from  the  time  the  Ignitor  la  removed  from 
its  box  until  the  round  la  fired. 

*  I  , 

xfi  »  Ignitor  resistance. 

x?  -  Ignitor  delay. 

Xg  -  0  if  the  measurements  were  obtained  on  inatnunent  table  #1. 

»  1  If  the  measurements  were  obtained  on  instrument  table  #2. 

x j ,  x  2  » . represent  the  different  strain  gages  used  to  measure 

pressure.  (/Vet ion  time  is  defined  as.  the  length  of  time  during 
burning  that  the  chamber  pressure  is  above  a  certain  amount.)' 

The  cross-products  of  Xj,  x3,  and  x^  were  also  used. 

For  a  particular  production  lot  studied,  these  variables  explained  nearly 
60  per  cent  of  the  total  variability  of  the  action  time  measurements.  When 
the  residuals  of  this  analysis  were  further1  related  to  production  variables, 
about  half  of  the  remaining  variability  was  accounted  for. 

There  is  nothing  new  about  using  regression  analysis  but  the  main  reason 
for  discussing  it  here  is  to  show  that  the  measurement  and  tasting  varia¬ 
bility  can  sometimes  appear  in  the  explained  sums  of  squares  as  wall  as 
the  residual.  Another  reason  for  mentioning  this  technique  Is  tore-echo 
some  of  the  warnings  diet  have  been  sounded  in  the  past  regarding  the  misuse 
of  regression  analysis.  When  regression  analysis  first  appeared  on  the 
scene.  It  was  so  overworked  that  many' examples  of  "nonsense"  correlations 
began  to  appear  end  this  necessitated  the  development  of  partial  regression 
and  correlation  methods.  After  that,  regression  analysis  became  a  little 
more  respectable  and  more  reputable  results  began  to  appear.  I  feel  that 
with  the  advent  of  high  speed  computers  (the  above  problem  takes  about 
a  minute  on  a  700-Series  IBM  Computer)  we  are  approaching  another  pro¬ 
blem  era.  It  is  relatively  easy  to  include  a  large  number  of  independent 
variables  and  this  may  tempt  people  to  throw  in  everything  but  the  kitchen 
sink  and  in  every  Inconceivable  combination.  Eventually,  the  Type  I  errors 
con  get  so  large  that  erroneous  results  can  almoin  be  guaranteed.  Also, 
this  "kitchen  sink"  technique  is  apt  to  decrease  the  number  of  degrees  of 
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freedom  associated  with  the  residual  rather  rapidly  so  that  high  correlations 
can  result  solely  because  the  number  of  parameters  fitted  is  nearly  as  large 
as  the  number  of  observations.  There  Is  no  easy  way  out  of  this  but  I  feel 
that  it  is  well  that  these  problems  be  mentioned  occasionally  to  prevent 
fingers  from  being  pointed  at  ua  collectively  again. 

An  obvious  criticism  of  this  example  Is  that  It  is  essentially  PARC 
analysis  and  we  arc  being  continually  advised  that  this  is  not  the  way  to 
do  things.  There  are  a  number  of  valid  reason's  why  PARC  analysis  should 
not  be  used,  not  the  least  of  which  is  that  a  particularly  important  factor 
may  not  very  much  over  the  period  of  time  represented  by  the  data  ahd  con¬ 
sequently  may  not  appear  to  be  very  important.  However,  in  ballistic  mis¬ 
sile  testing,  one  cannot  conduct  designed  experiments  -  the  present  testing 
procedure  is  expensive  enough  -  and  hence  we  must  make  do  with  what  we 
have.  Fortunately,  in  this  case,  the  primary  goal  was  to  obtain  measures 
of  product  and  testing  and  measurement  variability  rather  than  to  obtain  a 
functional  relationship  among  the  measurement  variables  and  the  test  data. 

3*  Principal  Components.  The  method  of  principal  components  has  been 
around  a  long  time.  Karl  Paarson^  suggested  this  technique  around  the  turn 
of  the  century.  Hotelling^  developed  methods  for  its  use  in  the  mid-thirties 
and  this  technique  coupled  with  factor  analysis  has  kept  many  psychome- 
trlcians  occupied  for  some  time.  Only  in  the  last  ton  years  has  much  use 
been  made  of  principal  components  in  industry  where  it  has  been  used  as  a 
control  tool7  and  a  method  of  prediction  as  well  as  its  primary  use,  the  . 
determination  of  the  structure  of  a  system.  As  somewhat  of  a  by-product 
of  this  technique,  we  may  also  use  It  to  obtain  an  approximate  method  for 
separating  product  from  measurement  variability.  This  can  be  illustrated 
by  a  specific  example: 

In  the  static  testing  of  ballistic  missiles  such  as  the  Nike  booster,  for 
a  characteristic  such  as  total  Impulse,  four  actual  measurements  are  made 
on  each  round  during  a  test  firing.  Each  round  has  two  thrust  gages  attached 
to  It  and  each  of  these  gages  Is  In  turn  recorded  on  both  an  oscilloscope  and 
an  electronic  integrator.  We  can  then  treat  this  as  a  four  variable  problem. 
Since  these  variables  arc  highly  correlated,  one  would  expect  that  the  first 
characteristic  vector  associated  with  a  covariance  matrix  of  these  variables 
would  represent  product  variability  and  the  remainder  might  give  some  in¬ 
sight  into  the  measurement  errors.  Since  the  sum  of  the  characteristic  roots 
associated  with  these  vectors  equals  the  trace  of  the  covariance  matrix,  the 
trace  may  be  assumed  to  he  t  rough  measure  of  the  total  variability  of  the 
system.  The  ratio  of  each  root  to  the  total  may  then  be  considered  a  measure 
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of  the  variability  explained  by  that  particular  principal  component.  It 
should  be  emphasized  that  this  method  can  be  best  employed  if  the  original 
variables  ere  all  in  the  s»e-.a  units  and  have  the  same  variances  although 
lor  prediction  purposes,  this  requirement  is  not  necessary. 

A  particular  example' using  Nike  booster  data  has  been  discussed  In 
Technoratrics  .  The  first  vector,  as  might. be  expected,  represented,  es¬ 
sentially,  the  average  of  the  lour  readings  and  explained  78  per  cent  of 
the  trace.  The  other  three  vectors,  accounting  for  22  percent  of  the  trace 
represented  gage  differences  and  integrator  vs.  oscilloscope  differences.  - 
One  should  not  infer  that  78  per  cent  of  the. total  variability  was  product 
variability  since  this  component  actually  represents  variability  common  . to 
all  four  measurements  which  would  consist  not  only  of  product  variability 
but  soiav  of  the  factors  mentioned  in  the  regression  example.  One  could 
Infer  however,  that  approximately  22.  per  cent  of  the  variability  of  Individual 
measurements  could  be  attributable  to  instrumentation  variability.  The 
first  transformed  variate  could  then  be  used  as  a  starting  point  for  other 
studies. 

4.  TTv?  Methods  of  Crubbs.  Trutk-rl  and  David*.  Quite  often  in  industrial 
work,  one  obtains  two  sets  of  data  which  are  functionally  related  In  some 
way.  Quite  often  this  will  involve  duplicate  measurements  on  a  series  of 
items.  If  both  measurements  are  made  with  the  same  equipment  and  per¬ 
sonnel,  the  data  are  commonly  analysed  by  a  one-way  analysts  of  variance, 
the  "between"  sum  of  squares  representing  the  variability  among  the  items 
themselves  (and  changes  in  level  of  the  equipment  if  any  such  exist)  end 
the  "within"  sum  of  squares  representing  measurement  variability.  On  the 
other  hand,  one  of  each  pair  of  observations  can  be  made  with  one  piece  of 
equipment  and  the  second  observation  on  another.  If  theoe  same  two  pieces 
of  equipment  are  employed  for  a  series  of  items,  then  the  duplicate  var¬ 
iability  may  not  necessarily  be  entirely  random  since  a  bias  may  exist 
between  the  two  pieces  of  equipment.  Data  of  this  sort  are  commonly  ex¬ 
amined  by  means  of  a  randomized  block  analysis  with  the  "treatments"  being 
the  items,  the  "blocks"  being  the  pieces,  of  equipment  and  the  residual 
representing  the  inherent  variability.  (Actually,  the  residual  is  an  Item  x 
equipment  interaction. but  In  most  cases  this  can  be  considered  inherent 
variability.)  These  two  situations  represent  the  most  widely  used  approaches 


*  These  methods  were  first  proposed  for  ballistic  missile  static  tests  by 
B.  £.  Thomipiron,  Allegany.  Stub  Hines  Laboratory^. 
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in  industry!©  the  separation  of  product  and-  measurement  variability.  The 
second  of  these  situations  deserves  a  bit  more  consideration,  however. 

When  the  duplicate  measurements  are  made  on  different  pieoea  of  equip¬ 
ment  or  by  different  persons,  there  is  no  guarantee  that  the  inherent  varia¬ 
bility  will  be  the  same  for  both  sets  of  data.  This  is  one  of  the  basic  assump¬ 
tions  of  the  analysis  of  variance  although  we  are  now  consoled  with  the  con¬ 
clusion  that  the  violation  of  this  assumption  Is  not  too  important.  Neverthe¬ 
less,  if  the  inherent  variabilities  of  the  two  sets  of  data  are  different,  the 
industrial  statistician  might  want' to  know  what  they  are.  Two  possibilities 
arise: 

Case  I.  Mean  values  of  the  two  sets  of  data  are  the  same  or  differ  by  a 
constant.  (Grubbs'  method.}2  .  .• 

Let  yjj  represent  the  1-th  measurement  on  the  1-th  item.  This  cap  ' 
be  expressed  in  the  following  manner; 


yt|  *  Xi  +  °\  4  Blj 

where is. the  overall  mean,  is  the  deviation,  of  the  l-th  item  from 
the  mean,  (5j  is  the  bias  of  the  J-th  piece  of  equipment  and  *y  the  in¬ 
herent  variability  associated  with  the  Ij-th  measure  me  ntv  Xj  and  e^  are 

random  variables  and  suggest  the  following  variance  component  models  for 
the  two  pieces  of  equipment: 

2  2  2 

or.  -  cr.  .  Cr^ 


Under  the  assumption  ttel  and  are  uncorrelated,  the  estimates  of 
these  components  are  obtained  as  follows: 
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s2  -  * 
x  * 1  y2 


.2  2  .  . 

*-  *  «„  '  * 


yly2 


■  * 


*2 


-  s 


*1*2 


Example: 


£nv-\'I 


‘K-y 


\$0\ 


■& 


fc.V 


In  the  atatia  testing  of  ballistic  missiles,  two  strain  gages  ere  employed 
to  nwasura  each  of  tha  attributes  of  pressure  end  thrust.  The  information 
from  each  of  theso  gages  is  then  relayed  back  through  an  electronic  system 
with  the  final  rasult  appearing  either  on  an  electronic  integrator  or  an  os¬ 
cilloscope  as  wo  have  already  mentioned.  The  present  example  consists 
of  the  results  on  thirty-seven  pressure  measurements  of  Nike  boosters  all 
from  one  production  lot  and  conditioned  at  130°  F.  Two  observations  (P- 
S.I.  units)  have  been  obtained  for  each  round,  one  front  the  integrator  re¬ 
lated  to  each  pressure  gage  so  the  present  method  will  be  employed  to 
estimate  the  inherent  variabilities  of  each  measurement  system.  The  var¬ 
iance  of  each  set  of  data  and  the  covariance  were  found  to  be: 


{•  ;vS-Sj 

IL. . J 


*k’\  v «, 

i  J*» 


•  *  *.  «*  W_ 

I  „*•  *k*  I 


o 


a2  -  606,  s2  “  605  and  a  ■  580. 
yi  y2  yly2 


-  \ 


The  "product"  variability  is  then  a*  »  sy  y  «  580.  The  measurement 

7l72 

variabilities  are  given  by 

•n  ■  \  '  \ft  ‘  26  ,nd  **2  "  ■  *  Vi  '  “• 


-  25 
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From  this  It  can  be  seen  that  the  inherent  veri abilities  of  the  two  systems 
are  about  the  same.  (Had  a  difference' existed,  a  number  of  reasons  could 
be  suggested  such  as  a  difference  in  the  variability  of  the  gages  used, 
number  of  tubes  replaced  in  the  system,  etc.)  Furthermore,  it  can  be  con¬ 
cluded  that  the  inherent  variability  is,  on  the  average,  about  one-sixth  of  • 
the  total  variability.  The  remainder  cannot  be  considered  solely  product 
variability  since  it  has  already  been  suggested  in  the  earlier  examples  . 
that  such  things  as  temperature  affect  the  overall  results  of  both  gages 
and  hence  s  ^  might  be  considered  a  measure  of  product' and  testing  var- 
iabillty.  As  stated  in  the  introduction,  no  one  of  these  methods  will 
singlehandedly  resolve  the  problem  of  completely  separating  these  var¬ 
iabilities.  A  comprehensive  study  would  probably  involve  the  use  of 
several  cf  these  techniques. 

It  should  be  pointed  out  that  Grubbs  has  alao  derived  methods  to  handle 
triple  and  quadruple  measurements  although  the  mechanics  are  not  as  simple 
as  the  ones  shown  here. 

« 

Case  II.  friean  values  of  the  two  sets  of  data  have  e  fixed  ratio. 

(The  method  of  Kruskal  and  David.) 

This  problem  must  be  stated  in  a  slightly  different  manner  since  it. can 
cover  a  wider  range  of  coses.  Again,  we  have  two  sets  of  data  but  thets 
need  not  be  the  same  type  of  measurements  on  each  item.  They  may  be  • 
duplicate  measur  ements  or  they  may  be  related  measurements  such  as 
pressure  and  thrust.  Since  they  nay  be  different  types  of  measurements, 
the  model  now  changes  to: 


YU  "  *  j  *  *U  4  % 


where  /*  <s  the  mean  of  the  J-th  type  of  measurement,  xjj  is  the  devi¬ 
ation  of  the  J-th  measurement  of  the  t-th  item  from  its  mean  and  e^ 
is  the  inherent  variability  associated  with  the  | -th  measurement  on  thp 
1-th  item.  In  relating  this  case  to  the  model  in  Case  1,  M. j  «  JX  4(5^ 

except  that  6 ^  Is  not  necessarily  a  bias  and  x^  is  not  necessarily 
equal  to  Xj2  as  was  the  case  previously.  The  Xy  are  random  but  no 
longer  independent  being  restricted  by  the  relationship: 

*1  4  xil  ^  c 

J*2  *  -Xtf 
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where  £  is  a  fixed  and  known  constant  (known  In  the  sense  of  any  parameter 
we  may  still  have  to  estimate  it)« 

■  For  any  Individual  item,  any  variation  of 

yll  "  -*i  +  X11  4  V 
y12  >»2  4  *12  4  *12 

from  £  then  is  due  to  and  e12  and  from  this  it  is  possible  to  obtain 

2  2  2  2  I 

estimates  of  9\  and  as  well  as  or  \  and.cr*  .  These  astlH 

*1  e2  *1  2 
mates  can  be  obtained  from  the  following  relationships: 


s  .  -  s  (i.e.  errors  are  uncorrelated) 

yly2  *lx2 


2  2 

Therefore,  ell  of  the  estimates  can  be  obtained  from  s*  ,  s  „  ,  sw  _ 

*  Y\  Y2  yV2 


(as  in  Casa  I)  and  a  knowledge 


of  £.  If  f  -1. 


we  hava  Case  I. 


Example: 


This  "•ftcb/Uque  has  ijnw.it  pmpvwvnd  es  a  rnsrhod  to  obtain  ur tea  suras  of  the 
variability  for  thrust  and  pressure  measurements  using  the  a r.sum ptlon  that 
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the  ratio  of  the  time-integrals  for  thrust  and  pressure  is  constant  for  a  given 
production  lot.  From  the  same  let  mentioned  in  the  previous  example  the 
time  integrals  {averaged  over  the  two  gages)  v/ere  obtained  for  thrust  and. 
pressure.  Since  the  true  ratio  of  these  quantities  varies  from  lot  to  loft,  it 
is  necessary  to  estimate  £  from  the  data  itself.  If  y^  designates  Jr  At 

and  y ^  designates  Jpdt,  then  yj  *=  H81S2,  .y^  *  32.76,  £  *  45.23, 

s2  *  385752.25,  s2  *  '  543.26-  and  s  -  4975.66.  From  this, 
yl  H  yly2 

s2„  *’22504.91  ,  si  -  110.01.  s2  *  16C703.15  and ' s*  -  433.25. 
xl  *2  1  2 

From  these  results,  it  would  a.ppe-ir  that  roughly  40  per  cent  o!  the  total 
variability  of  thrust  measurements  and  PO  per  cent  of  the  pressure  variabil¬ 
ity  could  be  attributable  to  measurement  variability. 

Two  points  should  be  mentioned  in  connection  with  this  example: 

V.  -Although  the  proportion  of  measurement  variability  seems- fairly 
high,  the  coefficients  of  variation  for  the  total  variability  of  these  measure¬ 
ments  are  of  the  order  of  four-tenths  to  seven-tenths  of  one  pet  cent. 

2.  There  has  been  considerable  evidence  to  indicate  that  the  errors 
•in  thrust  und  pressure  muasfurcments  may  be  correlated  which  of  course  would 
invalidate  the  use  of  this  technique  to  solve  problems  of  this  type  until  It 
ha 3  been  modified  to  allow  for  correlated  errors.  The  method  Itself  Ib,  how¬ 
ever,  perfectly  valid  as  long  as  the  model  holds. 

II.  ACCEPTANCE  SAMPLING5  ■ 

1.  Introduce  on.  The  motivation  for  this  part  stems  from  the  acceptance 
sampling  programs  used  in  the  evaluation  of  production  lots  of  ballistic  mis¬ 
siles  such  as  the  Honest  John  and  the  Nike.  These  particular  missiles  are 
operational  and  have  been  on  a  production  basis  for  some  time.  They  are 
currently  produced  in  lots  and  subjected  to  the  ordinary  acceptance  sampling 
schemes  used  in  quality  control.  Since  the  testing  of  these  missiles  Is  very 
expensive,  judgments  on  lots  should  be  made  with  as  little  inspection  as 
possible  consistent  with  the  prescribed  risks  of  accepting  poor  lots  and  re¬ 
jecting  good  ones.  This  suggests  sequential  sampling  which  has  come  into 
widespread  use  in  the  past  few  years  and,  In  fact,  some  types  of  missiles 
are  now  inspected  in  that  manner. 

There  are  several  Important  parameters  to  be  Inspected  on  each  round  In 
on;1  .s  f'.-v  s uch  oituj,jcuh~;  sties  are  action  time.,  thrust  or 
impulse,  and  some  measure  of  chamber  pressure.  These  variables  are  Inter- 
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related  and  hence  the  problem  is  a  multivariate  one*  In  present  day  oper¬ 
ations,  separate  sequential  plans  must  be  set  up  for  each  parameter.  It  Is, 
therefore,  possible  to  get  conflicting  answers  about  the  quality  of  a  lot, 
sampling  may  terminate  Tor  one  characteristic  before  another  and  there  is. 
no  appreciation  of  the  true  sampling  risks  involved  in  the  overall  program. 

It  is  obvious  that  a  sequential  multivariate  technique  should  be  used.. 

In  this  article  wa  will  give  some  multivariate  sequential  inspection 
schemes  for  the  characteristic  averages  both  for  the  cose  where  the  popula¬ 
tion  covariance  mntrlx  is  known  or  assumed  to  be  known  (a  typical  qual¬ 
ity  control  situation)  and  where  it  must  be  estimated  from  the  rdjnole.  When 
the  covariance  matrix  is  known,  we  use  a  sequential  “X  -test;  when  the 
covariance  matrix  Is  estimated  from  the  Sample,  we  use  a  sequential  7  -test. 

2 •  Sequential  Univariate  and  .Multivariate  Procedures  for.Testlnq  Means. 

In  univariate  situations,  test  procedures  have  been  constructed  to  test  tha 
null  hypothesis 

H  Q  ;  M  -  (or  M  -  Q  «  c5) 


against  the  alternative 

.  Hj  j  -U  A  Mq  (or  M  -  Mq  ji  6). 


When  these  procedures  are  extended  to  the  sequential  case,  it  is  customary 
to  replace  these  with  more  specific  hypotheses,  viK 

H0  *  *  -  Ma  (orJI  - 

Hj  ;  M  -  JJ1  (or  JJ  -  Mq  ~  6  i). 

Tor  the  case  where  the  population  variance  is  known,  the  sequential  proced¬ 
ures  have  been  worked  out  by  Wald  and  for  the  case  where  the  variance  is  not 
known,  by  Wald,  Rushton  and  others. 
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In  the  multivariate  case,  these  expressions  could  be  replaced  by  p -var¬ 
iate  vectors.  The  null  hypothesis  could  be  given  as 


H  «  ji  ■  M 
0  *  — -  0 


■‘V'-V-V-V 

.  - 
’  s  S  t  *  M  •  ■ 

*  ->V-V.V 
V-V-\' 

. > 

i  itiO  >.Tj 


but  it  becotries.  very  difficult  to  specify  a  meaningful  single  alternative  since 
there  are,  presumably,  infinitely  many  points  in  p-space  that  are  of  equal 
Importance  as  alternatives  and  even  a  hypothesis  of  the  type 


sr 


would  be. difficult  to  specify.  It  is  easier  to  operate  with  the  surfaces  of  p- 
dimenslonal  ellipsoids.  For  instance,  the  statements  M.  *  jU  Q  and  {M  - 

llj]  x  Jl  1  -  -^q)'  ■  0  are  identical  but  the  quadratic  form  of  the 

latter  expression  can  also  be  set  equal  to  seme  scalar  quantity  viz: 


<*.-•*<>>  <*.-  v  -  K 


which  represents  the  entire  surface  of  a  p-dlmensional  ellipsoid  while  the 
expression  M  -  Wn  ■  would  represent  only  ono  point.  Similarly, 
the  alternative  hypothesis  would  be  of  the  same  form  but  equal  to  a  larger 
scalar  value.  Our  hypotheses  become 


w 

V-V-I 
/v",-. -*1 


HqJ  (JU  -  AlQ)  E“l  (*»  -  W  Q)'  -  \  *  (quite  often  zero) 


Hj*.  lii  -MA  Z  ~l  (U  -  M0)‘  -  A*  (\j>^0). 


m- 

VV- 

V.-.V-V- 
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3 .  Test  Procedures.  Although  the  form  of  the  sequential  procedures  differ 
for  the  case  £  known  or  £  unknown,  they  are  quite  similar  la  admlnis-  ■ 

tration.  All  sequential  procedures,  in  the  Wald  sense,  employ  a  sequential 
probability  ratio  Pjj/  P0n  which  is  evaluated  after  each  new  observation 

is  taken.  Let  Oc  and  $  denote  the  usual  Typo  I  and  Type  II  errors.  If, 
after  n  observations  have  been  made,  Pj^/P^  —  $/U  -tt)  we  accept 

V  lf  Pin/P0n  -  (1  -  p  )/«L  we  accept  Hj!  .if  J3  /ft  -«**)  <  Plr/P0n  < Xi-JDAt 

we  iinfer  that  we  do  not  have  enough  Information  dnd  prddeed  to  take  another 
observation  and  repeat  the  entire  procedure. 


& 

m 

M: 


X2  - 


If  the  population  covariance  matrix  Is  known,  we  have  the  sequential 
-test  with 


PJn/POn  ‘  •""|A  1  '  A°’/2  “A1  AV4)<P1(P/J!  "XSX> 


where  x  is  a  p-elewent  vector  of  sample  means,  based  on  n  observations, 

X  *  n  (x  -  y n)  I.  (x  -  Z/_)’  and  F  (c;x)  represents  the  generalised 
n  ~  „  0 

hypergeometric  function: 


F(c;x)  *  !  +  4 


etc  ij  (c  + 


♦  . . 


I*"-.  '•>' 

It: k 


•*  0,  the  probability  ratio  reduces  to: 


pin /  p  on  ■  • A‘  n  or  i Ip/2;  n  \2  x-2/4>- 
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If  the  population  covariance  matrix  is  not  known  and  must  be  estimated 
from  the  same  sample  as  the  sample  means,  we  have  the  sequential  X  -test 
with 


p  /p  -e 
in  On 


n(Xl  "  \)/2  F  f  n/2.  p/2;  n^.2  T2/2  In-l-aT*)] 
'111  I  n  n  J 


•r  ,r,  T*/»b-i*$] 


where  S  represents  the  sample  covariance  matrix  based  on  n  observations, 
T2  »  n  lx  -  U  )  S-1  (x  -  JU  )  and  T  .  la,  c;  x)  represents  another 

generalized  hypergeometrlc  function 

p  la  c-  x)  -  i  ♦  ♦  gfe  t  Uit2--  +  o(a  *,}\  fr  +  ♦. 

!F1  {,C,)  a  c(c4U2J  c(c  +  1)  (c  ♦  2)3 1* 


which  is  more  familiarly  known  as  the  confluent  hypergeometrlc  function.  If 
\  *  *  0,  the  probability  ratio  reduces  to: 

Von””*1  n  /,  - 

Both  of  these  procedures  terminate  with  probability  unity  and  the  risks  of  in¬ 
correctly  accepting  Hj  and  HQ  are  approximately  equal  to  OL  and  p  res¬ 


pectively. 
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rejected  when  the  true  means  of  all'  three  characteristics  are  on  standard.  It 
will  further  be  assumed  that  only  a  meager  amount  of  information  is  available 
concerning  the  variability  of  these  characteristics.  From  this  information,  it., 
is  inferred  that  the  individual  tolerances  constitute  limits  of  £,.3er  for  in¬ 
dividual  observations  about  their' standards  and  that  there  is  no  evidence  that 
the  variables  are  correlated.  (The  assumption  of  independence  regarding  .  ‘ 
tolerances  ia  not  always  valid.  In  some  types  of  missiles,  total  impulse 
and  action  time  must  be  negatively  correlated  to  Insure  a  fixed  range  for  their 
flight.) 

Considering  each  characteristic  separately,-  the  requirement  that  97.5 
per  cent  of  the  lot  must  be  within  tolerances  implies  that  the  true  lot  mean 
for  that  characteristic  cannot  be  closer  than  2.24 cr  to  either  tolerance 
limit;  conversely,  the  true  mean  must  be  within  .76  cr  of  the  standard  since 
the  tolerancOG  were  assumed  to  be  +3 a  limits.  Several  possibilities  exist. 
One  possibility  would  involve  Inscribing  an  ellipsoid  in-glde  the  rectangular 
solid  bounded  by  ju  t  +  .75 c^;  i  *  1,  2,  3.  This  would  be  rather  restric¬ 

tive  andwould  be  employed  In  the  case  where  the  specifications  were  to  be 
strictly  employed.  The  nonroentrality  parameter  under  T for  a  given 

2  ‘ 

characteristic  would  be.  (.76)  .  and  fcincetthe  occurence  of  an  obt-of-s pacifi¬ 
cation  condition  for  any  one  of  the  variables  is  sufficient  reason  to  reject 
the  lot,  A.  2  *  .5776.  Considering  the  crudeness  of  the  determination  of 
>2  *  2 

A  in  the  first  place,  a  value  of  -  .5  is  probably  quite  adequate. 

Our  hypotheses  have  been  restated  as; 

H  ;  \  2  »  o 
0 


Since  the  covariance  matrix  is  unknown,  we  now  employ  the  sequential  y-V 

2  v v  *-  • '  - 

T  -test.  On  the  average,  we  may  expect  to  test  27  rounds  if  Hq  is  true 

or  24  rounds  If  Hj  is  true  whereas  the  corresponding  fixed-sample-slxe  \ 

fest  wmuld  require  37  rounds.  The  fact  that  so  many  rounds  are  required  in  / 

the  ejuiTtple  indicates  that  this  procedure  is  considerably  more  exacting  than  m 

J 
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the  current  methods  now  employed  or  conversely  that  the  value  ef  X  *  Implied 
by  current  procedures  is  considerably  larger  than  .5. 

A  second  method  would  be  to  circumscribe  an  ellipsoid  around  the  rect¬ 
angular  solid  representing  the  specifications.  This  Is  note  conservative 
than  the  other  method  and  would  result  in  more  material  being  accepted.  This 
method  can  be  used  when  the  lot  would  be  acceptable  even  If  all  of  the  char¬ 
acteristics  were  borderline.  This  would  yield  a  value  of  X  2  ■  3(.7S)  *1.73. 

Rounding  to  a  value  of  X  2  *2.0,  this  procedure  would  require,,  on  the 

average,  6  rounds  to  reach  a  decision  under  Hq  and  10  rounds  under  Hj 

as  compared  with  a  fixed-sample  size  of  13.  This  should  demonstrate  quite 
adequately  the  problem  associated  with  specifying  the  hypotheses  in'muulti- 
variute  analysis.  Cther  possibilities  for  setting  up  these  hypotheses  could 
Involve  acceptance  sampling  for  variables  techniques  or  sequential  estimation 
procedures  but  such  techniques  have  not  yet  been  developed.  . 

Case  II:  E  known 

We  wlll-i*ow-c3«utnft-that4n4ho  tlsno  which  hae-erlapsed  since  the  oper¬ 
ations  carried  on  in  the  preceding  section,  sufficient  information  has  been 
gathered  so  that  the  population  covariance  matrix  can  be  assumed  to  be 
known..  We. may  now  use  a  sequential  7C  2'.-»te3t.  .  Suppose  thdt  It  turned  out 
that  the  variances  of  these  three  variables  were  smaller  than  originally 
supposed  so  that  the  original  tolerances  were  larger  than  +3  or.  This  sug¬ 
gests  several  possibilities  again.  One  possibility  would  be  to  use  the  nat¬ 
ural  tolerances  of  the  process  and  allow  A  2  to  remain  at  .5.  This  pro¬ 
cedure  is  often  employed  when  the  acceptance  sampling  program  is  also  used 
to  control  the  process  but  in  the  case  of  Nike  boosters,  too  much  materiel 
of  acceptable  quality  would  be  rejected  and,  considering  the  cost  factor, 
this  would  not  be  a  recommended  procedure. 

Other  possibilities  would  involve  Inscribing  or  circumscribing  an  elllpaold. 
based  on  the  now  known  covariance  matrix  about  the  tolerances.  When  var¬ 
iables  are  highly  correlated,  circumscribing  can  lead  to  acceptance  of  a  fair 
amount  of  unsatisfactory  material.  Suppose  that  by  inscribing  an  ellipsoid, 
we  arrive  at  a  value  of  K  2  -  1,0.  A  sequential  li  2 -test  of  this  type  . 

would  require,  on  the  average,  13  rounds  under  and  §  rounds  under 

Hj  compared  with  a  fixed-sample  size  of  IB. 
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5.  Computations  t  It  would  be  only  fair  to  state  that  the  computational  re¬ 
quirements  for  the  sequential  T^-test  am  not  modest  since  the  sample  co- 
variance  matrix  must  be  inverted  for  each  observation.  However,  in  ballistic 
missile  testing,  this  cost  is  still  negligible  when  compared  to  the  actual 
oost  of  the  round  itself.  The  computations  for  the  sequential  X  ^-test  are 
quite  straight-forward  requiring  only  vector  by  matrix  multiplication. 

6.  Generalized  9C  2 -statistics.  An  additional  technique  which  may  be  em¬ 
ployed  for  Case  II,  the  situation  where  the  covariance  matrix  is  known,  in¬ 
volves  the  use  of  the  generalized  "X  2 -statistics  developed  by  Hotelling. 

This  allows  us  to  compare  not  only  the  sample  mean  of  a  lot  with  the  standard 
but  also  the  covariance  matrix  of  the  sample  with  the  previously  established 
covariance  matrix.  It  is,  of  course,  possible  for  the  mean  of  the  lot  to  be 
close  to  standard  but  for  the  variability  of  individual  rounds  to  be  exces¬ 
sive  enough  to  impair,  the  overall  quality  of  the  lot  anyhow.  Techniques  are 
now  available  to  test  sequentially  the  mean,  covariance  matrix,  and  if  ap¬ 
propriate,  the  overall  variability  of  the  lot,  although  this  last  test  is  not 

os  discriminating  as  the  others  and  by  the  time  this  test  had  rejected  a  lot 
one  of  the  other  two  would  have  probably  already  rejected  it. 

7.  Acknowledgements .  The  research  for  the  Part  II  of  this  paper  was  spon¬ 
sored  by  the  Office  >of  Naval  Research^  Department  of  the  Navy:  Contract 
Number:  NONR-235'2(01),  Task  Order  NR  042-019  with  the  Virginia  Polytechnic 
Institute,  Ralph  A.  Bradley,  Principal  Investigator.  An  article  Illustrating 
these  techniques  with  numerical  examples  based  on  Radford  Arsenal  data  is 

in  preparation  and  should  appear  shortly. 
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A  TRIAL  COMPARING  CERTAIN  SIDE  EFFECTS  OF  TWO  NERVE  GAS 
ANTIDOTES,  USING  HUMAN  SUBJECTS 

C.  A.  daCandole,  MD* 

Defence  Research  Medical  Laboratories 
Downtvlew,  Ontario,  Canada 

and 

B.  A.  Richardson 

Canadian  Army  Operational  Research  Establishment 
Ottawa,  Ontario,  Canada 

INTRODUCTION .  This  paper  describes  the  experimental  design  of  a  trial 
performed  at  Camp  Borden,  Ontario,  in  February  of  I960,  with  the  cooper¬ 
ation  of  the  Canadian  Forces  Medical  Service,  to  compare  certain  side  af¬ 
fects  of  two  nerve  gas  antidotes,  using  human  subjects.  It  must  b«  made 
clear  at  the  outset  that  the  human  subjects  used  were  not  exposed  to  nerve 
gas,  since  interost  lay  only  in  the  side  effects  of  the  drugs  under  test. 

Curtain  particulars  of  the  drugs  and  dosages  used  In  the  trial  and  the 
numerical  results  obtained  are  security  classified.  In  the  present  paper, 
therefore,  reference  to  these  topics  will  he  made  in  cod  ad  or  qualitative 
terms. 

The  accepted  common  nerve  gas  antidote,  atrootno,  tends  to  Induce 
undesirable  side  effects  when  used  in  the  rather  high  dosage  levels  recom¬ 
mended.  These  offocts  include  blurring  of  vision,  nausea,  disturbance  of  • 
the  pulse,  dizziness,  and  a  tendency  to  faint  on  sudden  rising  to  the  feet: 
all  of  which  are  clearly  serious  defects  from  the  military  point  of  vlaw. 

In  Canada,  C,  A.  d&Cnndole  has  investigated  a  treatment  that  showed 
promise  cf  both  enhanced  protective  action  against  nerve  gas  as  well  as 
reduced  side  offecta,  when  tested  in  the  laboratory  on  animals  and  on  a 
small  number  of  human  volunteers.  By  1959  research  hod  rcecbed  a  point 
where  a  test  on  humans  (for  side  effects)  in  a  trtal  on  a  fairly  large  scale 
seemed  worthwhile.  In  that  trial,  a  modified  form  of  the  conventional  at¬ 
ropine  treatment  (here  called  "Treatment  A")  was  compared  with  da  Candole’a 
treatment  (here  called  "Treatment  B"). 


*  Now  at  Suffleld  Experimental  Station,  Ralston,  Alberta,..  Canada. 
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AIM.  As  relatively  little  was  known  about  Treatment  B,  it  was  thought 
prudent  first  to  learn  something  about  the  basic  physiological  consequences 
of  its  use  before  attempting  to  assess  the  effects  of  the  two  treatments  upon 
military  performance  directly.  A  limited  aim  was  therefore  established, 
namely: 

"To  compare  the  physiological  effects  of 
Treatments  A' and  B-.- 

KEf7QN5E  MSTAM3TSP.5 ,  The  physiological  effect*  to  be  recorded  ware  a's 
Follows.  First,  obviously,  any  visible  reactions;  not  only  instances  of 
fainting  but  also  the  less  dramatic  a  if  acta,  If  any  appeared — tremor,  rest¬ 
lessness,  pallor,  flushing,  end  so  on.  Next,  subjective  effects— dizziness, 
nausea ,  headache,  thirst,,  for  example.  1£  these  were  not  revealed  by  the 
te Ft  subject  spcntanoou3ly,'they  were  to  be  disclosed  by  direct  questioning. 

But  qualitative  end  subjective  data  would  net  be  enough:  quantitative, 
objective  data  wars  needed  uc  well.  New,  It  happens  that  some  of  the  side 
effects  of  military  importance  are  associated  with  disturbance  of  the  cardio¬ 
vascular  G/siem—the  heart  und  blood  vessels.  Fainting,  for  example,  'can 
occur  If  the  pulse  proflmire  falls  too  lew.  It  was  therefore  relevant  to  re¬ 
cord  the  blood  pressures  end  pulse  rates,  defined  as  follows: 

a.  The  Systolic  Blood  Pressure  Is  the  peak  pressure  reached 
during  the  contraction  of  the  heart. 

b.  The  Diastolic  Blood  Pressure  is  the  low  during  relaxation 
of  the  heart  while  It  is  refilling. 

c.  The  Pulse  Pressure  Is  the  difference  between  the  first  two. 

d.  The  Pulse  Rata  is  the  number  of  systolic  peaks  per  minute. 

Three  aspects  of  these  four  physiological  parameters  were  of  interest, 
namely: 

a.  their  absolute  values; 

b.  thotr  response  to  the  drugs;. that  is,  their  departure  from  normal 
after  treatment; 

c.  the  differential  resnont-g;  that  Is,  the  difference  between 
the  res  pause  to  A.  and  the  response  to  JJ* 
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EQUIPMENT.  The  physical  layout  of  the  facilities  provided. for  the  trial  is 
shown  in  Figure  1;  a  hospital  ward  with  sufficient  equipment  for  testing  8 
subjects  at  a  time,  with  one  M.D.  and  one.  assistant  at  each  testing  station 
to  act  as  observers  and  recorders .  Three  American  medical  doctors  partici¬ 
pated.  We  sre  pleased  at  their  'interest  in  the  trial  and  grateful  for  their 
help. 


JJ  0  0/0JT  JV-'; 

IAYOUT  OF  WARP 
Figure  1 

The  eight  testing  stations  were  arranged  four  along  each  wall,  with  a 
screen  running  down  the  center  aisle  so  that  the  tost  subjects  would  not 
directly  face  one  another.  At  each  end  of  the  room  were  tables  for  the  use 
of  the  Project  Officer  and  his  assistants  one  cf  whom  acted  as  timekeeper. 

Each  testing  station  was  outfitted  with  a  tilt  table,  for  the  teat  subject 
to  lie  upon.  Also  (not  shewn  in  Figure  1),  at  the  head  (outer)  end  of  each 
tilt  table  there  was  a  small  table  to  hold  instruments  and  other  equipment; 
and  at  the  foot,  a  drawing  board  with  blank  data -recording  forma,  and  an 
over-bed  table  for  other  necessary  equipment.  The  tilt  table  could  be  laid_ 
flat  or  tilted  quickly  upright  to  an  angle  of  85  degrees.  The  subject  was 
strapped  lightly  to  the  table  to  keep  him  from  pitching  forward  on  his  face 
on  tilting.  The  purpose  of  this  tilting  was  to  simulate,  in  a  standard  farb- 
ion,  the  act  of  suddenly  rising  to  the  feet. 


The  pulse  rates  were  recorded  by  means  of  the  electrocardiograph.  The 
electrocardiograph  is  more  accurate  they  digital  palpation,  and  U  gives  • 
permanent  record  that  caanibi  examined  at  leisure.  Blood  pressures  wees 
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measured  by  the  ordinary  sphygmomanometer,  since  no  satisfactory  in¬ 
strument  of  the  recording  type  was  available  at  Camp  Borden.  ■ 

CONTROL  OP  SOURCES  OF  VARIATION  IN  RESPONSE.  There  are  many  pos¬ 
sible  sources  of  variation  in  response,  as  biological  systems  can  be  ex-, 
tremely  sensitive  to  small  changes  in  the  conditions  towhiah  they  are 
subjected.  The  more  Important  disturbing  factors  in  the  present  instance 
fall  Into  four  categories;  those  dependent  on; 

a.  the  drugs  used; 

b.  the  test  subject; 

c.  the  environment; 

•  '  '  *  '  ‘  •  l  '  *  ,, 

d.  the  observational  technique. 

Tirst,  the  drug  factors:  there  are  at  least  four  of  these; 

a.  •  the  route  of  administration; 

b.  the  size  (volume)  of  the  dose  administered; 

c.  other  ingredients  in  the  formulation; 

d.  the  concentration  of  the  active  ingredients. 

The  effects  of  these  factors  were  eliminated  by  restricting  the  Bcape  of 
the  trial.  Both  treatments,  A.  and  Jit  were  administered  by  the  same 
one  route:  intramuscular  Injection.  The  dose  for  each  treatment,  that  is, 
the  quant,  y  of  active  ingredients  injected,  was  kept  constant,  not  varied 
in  proportion  to  the  subject's  body  weight.  Only  one  formulation  of  each 
mixture  was  tested.  Concentrations  were  chosen  so  that  the  volume  In¬ 
jected  Would  be  the  same  for  both  treatments. 

The  test  subjects  introduce  personal  factors  of  two  types:  fixed  and 
variable.  The  fixed  factors  include: 

a.  body  weight; 

b.  height; 


c. 


age; 


A 
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d*  normal  blood  pres  suras  and  pulse  Mte;  . 

e.  medical  history; 

1.  .  idiosyncrasies. 

The  variable  factors  Include: 

a.  time;  . 

b.  posture; 

c.  rate  of  tilt; 

d.  physical  state;  . 

e.  mental  state. 

Of  the  fixed  factors,  the  effects  of  body  weight,  height,  age,  and 
normal  levels  of  blood  pressure  end  pulse  rate  can  be  reduced  by  using 
matched  pairs  of  subjects,  giving  j\.  to  one  and  JJ_tothe  other.  Alter¬ 
natively,  one  can  use  the  same  subject  twice;  but  this  changes  his  med¬ 
ical  history  and  so  is  an  advantage  only  if  his  response  characteristics 
(hia  idiosyncrasies)  remain  unaffected.  A  "used"  man  might  conceivably 
give  a  worse  comparison  of  the  two  treatments  than  would  be  obtained 
using  a  second,  "frosh"  one.  So  to  get  the  advantage  either  way,  matched 
pairs  were  used  and  each  man  was  exposed  twice,  alternating  the  treat¬ 
ments. 

The  total  number  of  test  subjects  was  56.  The  personal  character¬ 
istics  of  this  group  were  relatively  homogeneous,  except  with  regard  to 
body  weight.  Body  weight  was  thus  the  major  difference  among  the  re¬ 
sulting  28  pairs. 

Time,  posture  and  rate  of  tilt  are  the  factors  whose  effects  were  under 
examination,  and  so  these  were  varied  deliberately. 

The  effects  of  hunger,  fatigue  and  other  physical  and  physiological 
states  were  minimized  by  restricting  the  free-time  activities  of  the  sub¬ 
jects:  special  meal  schedules,  no  heavy  exercise,  no  alcohol,  early  to 
bed.  To  control  mental  and  psychological  factors,  the  subjects  were  given 
a  thorough  briefing  in  advance  to  reduce  possible  apprehension  and  fear. 
Excitement  was  reduced  by  maintaining  an  atmosphere  of  calm  and  relax¬ 
ation  in  the  testing  ward. 
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Three  environmental  factors  can  be  identified: 

a.  '  the  climate  of  the  ward; 

b.  the  location  of- the  testing  station; 

c.  incidents  occurring  during  the  course  of  a  test  that  might 
affect  neighboring  subjects.  • 

These  factors  were  minimised  by  testing  the  subjects  in  groups  of  8  • 

(that  is,  4  pairs).  The  effects  of  any  change  in  the  general  climate  of 
the  ward,  temperature-,  humidity  and  noise  level  and  so.cn,  would  thus- 
show  up  in  the  variation  between  groups.  To  control  differences  between 
stations  the  subjects  were  assigned  at  random;  but  they  were- tested  fit 
the  same  station  on  both  test  occasions..  Sporadic  events  occurring 
during  the  trial  might  affect  response  but  they  would  surely  be  distributed 
at  random. 

The  observing  teams  were  assigned  to  stations  at  random/  at  the  be-, 
ginning  of  the  trial,  but  rotated  two  positions  for  the  second  round  of 
tests  so  that  they  would  not  handle  the  sams  subject  twice.  To  eliminate 
personal  bias  in  the  physicians — reading  blood  pressure  is  still  more  an 
art  than  a  science — they  were  drilled  in  a  standard  technique  and  required 
to  use  it  regardless  of  their  own  inclinations.  In  addition,  neither  they 
nor  anyone  but  the  Project  Officer  knew  the  identity  of  the  treatment  given 
in  any  instance. 

The  instruments  were  calibrated  before  the  trial.  Their  residual  var¬ 
iation  is  regarded  as  negligible. 

PROCEDURE.  During  the  course  of  the  trial  the  subjects  underwent  a 
series  of  posture  changes  as  illustrated  In  Figure  2:  horizontal,  vert¬ 
ical,  horizontal,  vertical,  horizontal.  Such  a  series  of  changes  will 
be  called  a  sequence.  Blood  pressures  ar.d  pulse  rate  were  recorded  at 
a  set  of  Intervals  within  each  sequence.  The  readings  and  posture  changes 
were  made  simultaneously  at  all  8  stations,  on  signal  from  the  timekeeper. 
The  same  schedule  was  used  for  all  sequences. 
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Each  subject  underwent  two  sequences  in  Immediate  succession-  The 
drugs  were  injected  at  the  end  of  the  first  or  central  sscuence.  thus  marking 
•the  sidrt  of  the  second,  or  drug  sequence.  The  pair- of  sequences -together 
constitute  a  session. 

The  55  test  subjects  required  7  sessions  in  all.  Every  subject  was% 
tested  on  two  occasions  separated  by  an  interval  of  2  days.  The  brder  of 
tasting  v/as  the  seme  on  both  occasions.  The  two  members  of  each  pair  or 
mates,  were  always  tested  together  in  .the  seme  session.  One  received 
Treatment  A  end  the  other  Treatment  B  on  each  occasion,  with  the  treat¬ 
ments  reversed  for  the  second.  The  mates,  therefore,  fall  into  two  classes 
according  to  the  order  of  treatment:  the  A-firsts,  or  A3‘s,  and  the  B-firsts 
or  BA's. 

TRIAL  DESIGN.  The  shape  of  the  trial, design  is  illustrated  in  Figure  3.  The 
basic  structure  consists  of  28  Latin  Squares  of  order  2  aranged  in  seven 
blocks  of  4.  One  such  block  is  shown  here.  Each  Latin  Square  represents 
one  pair  of  subjects.  Rows  represent  occasions,  columns  are  mates,  and  the 
treatments  occupy  the  diagonals.  In  the  third  dimension,  the  four  quadrants 
represent  the  sessions.  The  quadrants  could  be  shown  as  split  in  this  di¬ 
mension,  representing  the  control  and  drug  sequences,  but  if  we  confine  our 
attention  to  Response,  —  that  is.  Drug  minus  Control  —  the  split  aspect 
disappears  and  the  picture  is  as  shown  here. 


* 
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Figure  3 

*To  visualize  the  entire  trial,  the  other  six  blocks  can  be  imagined  as  re¬ 
ceding  into  the  background  behind  Block  1.  The  points  in  time  might  be 
.represented  by  rows  of  blocks  in  succession  left  to  right,  and  the  four  para¬ 
meters  thought  of  as  stacked  pne  on  top  of  another. 

RESULTS.  During  the- trial  two  test  subjects  fainted,  and  among  the  others  a 
number  suffered  reactions  to  Treatment  B  severe  enough  to  make  their  with¬ 
drawal  from  the  trial  advisoble.  As  a  result,  many  readings  were  lost--so 
many  that  use  of  a  missing -values  routine  was  out  of  the  question.  Anelysi*' 
was  therefore  confined  to  pairs  of  subjects  for  whom  complete  sets  of  readings 
were  available,  each  reading  time  being  considered  independently. 

As  an  example  of  the  kind  of  results  obtained,  the  responses  for  each  of 
the  four  physiological  parameters  at  the  time  cf  maximum  effect — Just  before 
and  Just  after  the  second  tilt  to  vertical — are  shown  in  Tables  l  and  2. 

These  represent  the  period  of  greatest  interest;  a  presentation  of  the  complete 
results  would  be  out  of  place  here.  The  tabular  entries  ore  the  results  of  £ 
tests  of  the  mean  and  of  each  of  the  four  main  factor  effects,  using  the  resid¬ 
ual  variance  as  error.  The  symbols  have  the  conventional  meanings.  For  sig¬ 
nificant  means,  the  direction  of  the  response  is  also  given,  plus  and  minus 
signifying  increase  and  decline,  respectively.  Slmllaily,  for  each  significant 
factor  effect,  the  level  shewing  the  greatest  response  in  the  same  direction 
as  the  mean  is  indicated.  is  the  number  of  (complete)  pairs  available  in 
each  case. 
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TABLE  1  , 

■  Effects  at  Time  of  Maximum  Response;  Subjects  Horizontal 


Contrast 

* 

Physiological  parameter 

Systolic 

BP 

(N-23) 

Diastolic 

BP 

(N-23) 

Pulse  Press. 
(N-22) 

Pulse  Rate 
(N-21) 

Mean  (vs  zero) 

**(+> 

**(+) 

*(+) 

**(♦) 

Treatment  (A  vs  3) 

**IB) 

**(B)  / 

,  **(B) 

NS 

Occasion  (1st  vs  2nd) 

NS 

NS 

'NS  ■ 

NS' 

Order  (AB  vs  BA) 

NS 

NS, 

NS 

NS 

Between  pairs 

** 

*>* 

NS'  ‘ 

TAB7JS  2 

Effects  at  Time,  of  Maximum  Response:  Subjects  Vertical 


-  Physiological  parameter 

Contrast 

Systolic 

BP 

(N=19) 

Diastolic 

BP 

(N=14) 

Pulse  Press. 
(N-14) 

Pulse  Rate 
fN-22) 

Mean  (vs  zero) 
Treatment  (A  vs  B) 
Occasion  (1st  vs  2nd) 
Order  (AB  vs  BA) 
Between  pairs 

**(-) 

**(A) 

NS 

NS 

NS 

**(+) 

**(B) 

NS 

ns  r 

NS  “ 

**(-) 

*(A) 

NS 

NS 

NS 

i 

**W 

**(A) 

**0) 

NS 

** 

CONCLUSIONS.  We  are  not  here  concerned  with  a  specific  physiological 
Interpretation  of  the  results;  IWs  sufficient  to  note  that  the  results  were 
unequivocal.  T.be  main  conclusions  were  as  follows; 
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a.  ■  Both  treatments  produce  definite  responses  in  subjects 
In  either  posture,  horizontal  or  vertical. 

b.  The  two  treatments  distinctly  differ  in  magnitude  of  response, 
except  in  that  of  pulse  rate  in  horizontal  subjects. 

Treatment  B  effectively  maintains  the  jaulse  pressure  on 
change  of  posture  from  horizontal  to  vertical,  as  intended, 
but  at  an  unacceptable  price  in  new  and  unforeseen  side 
effects. 

Day-to-dqy  differences  and  order  of  administration  of  the 
treatments  can  probably  bo  safely  disregarded  in  any  future 
trials  of  Treatment  B  or  modifications  of  it. 

There  was  significant  variation  among  pairs  of  subjects. 

Now,  this  .difference  represents  the  combined  effects  of  all . 
personal  factors  plus  any  variation  between  sessions.  Body 
weight  Is  probably  the  dominant  factor;  if  so,  the  recorded  . 
body  weights  should  account  for  most  of  the  difference  Ob¬ 
served.  This  portion  of  the  data  would  no  doubt  repay  fur¬ 
ther  analysis. 

SUMMARY.  This  paper' has  described  a  trial  performed  on  human  subjects  to 
compare  the  physiological  side  effects  produced  by  two  nerve  gas  antidotes, 
"Treatment  A"  and  "Treatment  B",  Treatment  B  was  designed  to  avoid  cer¬ 
tain  side  effects  that  tend  to  accompany  Treatment  A,  these  side  effects 
being  undesirable  from  the  military  point  of  view. . 

The  trial  clearly  showed  Treatment  B  to  be  superior  to  Treatment  A  in  the 
one  respect  of  special  intorest,  but  revealed  that  it  introduced  new  and  un¬ 
foreseen  side  effects  that  were  themselves  undesirable.  The  trial  thus  il~ 
lustrates.the  need  to  proceed  with  caution  in  complex  circumstances,  and  to 
examine  fundamentals  before  attacking  even  more  complex  problems  such  as 
the  evaluation  of  military  performance. 

This  trial  Is  presented  as  ar  example  of  the  usefulness  of  formal  experi¬ 
mental  design  and  analysis  of  variance  in  an  area  where  it  is  not  yet  regularly 
applied.  We  hope  Its  success  in  producing  some  clear-cut  answers  in  the 
presence  of  many  complicating  factors  will  help  spread  awareness  of  these 
valuable  techniques.  , 
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DESIGN  OF  AN  EXPERIMENT  FOR  THE  MOST  EFFICIENT 

„  CONDUCT  OF  SAFETY,  RELIABILITY  AND  PERFORMANCE  TESTS 

OF  FUZES  IN  THE  DESIGN  AND  DEVELOPMENT  STAGES  . 

Gertrude  Wetmraub 

Missile  Warhead  and  Special  Projects  Laboratory,  Picetinny  Arsenal 

PROBLEM.  To  design  ah  experiment  for  the  efficient  conduct  of  tests  to  de¬ 
termine  the  operational  and  safety  characteristics  of  fu2es  being  developed 
for  a  missile  warhead. 

STATEMENT  OF  PROBLEM.  Fuses  are  designed  to  accomplish  a  particular 
mission  in  the  successful  operation  of  ordnance  ammunition  such  ae  mines, 
warheads  and  missiles. 

Before  they  are  used  in  their  ultimate  mission,  they  are  subjected  to  var¬ 
ious  environmental  treatments  such  as  vibration  and  waterproofness  tests  to  . 
Insure  their  proper  functioning  and  safety  for  use.  During  the  conduct  of 
thesn  teats,  yes-no  responses  as  well  as  quantitative  measurements  are  ob¬ 
tainable.  Moreovor',  tests. are  generally  conducted  using  rather  small  size 
samples  to  determine  the  functioning  and  safety  characteristics  of  the  fuze  at' 
the  design  level.  Based  upon  the  results  of  these  tests,  performance  end  re¬ 
liability  estimates  ore  made.  In  addition,  engineering  Judgment  and  previous 
experience  are  also  usually  involved  in  making  reliability  estimates.  The 
fuze  Is  then  Judged  to  function  properly  and  to  be  safe  for  use. 

a.  Fuze  Design  Characteristics  • 

Fuzes  are  generally  composed ^>f  several  major  components  whose 
co-functioning  affects  the  overall  fuze  performance.  Also,  Incorporated  In  the 
fuze  design  are  various  safety  characteristics. 

t  *  * 

b.  Types  of  Environmental  Treatments 

The  various  types  of  environmental  treatments  to  which  fuzes  are 
likely  to  be  exposed  include  the  following: 

(1)  Transportation  Vibration  *.  * 

(2)  Rough  Handling 

(3)  Aircraft  Vibration 

(4)  Temperature  and  Humidity 
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(5)  Vacuum-Steam  Pressure 

(6)  Salt  Spray 

(7)  Waterproofness  ‘ 

.  (8)  Weathering  (Exposed) 

(9)  Jolt  and  Jurobl*  ,  , 

(10)  Low  Drop 
'(11)  Detonator  Safety 

c.  Evaluation  Teats 

These  measure  the  functioning  and  safety  of  the  fu2e»  alter  they 
have  been  exposed  to  the  environmental  tests.  They  include  the  following: 

(1)  Inspection 

(2)  Self-Destruction 

(3)  40  Ft.  Drop 

(4)  Functioning 

pnnrnssn  STATISTICAL  PROGRAM  TO  BTB  IMPLEMENTED..  The  following  types 
of  statistical  programs  are  currently  being  used  at  Picatlnny  Arsenal  in  the 
development  of  fuzes.  The  first  is  a  factorial  experiment  designed  to  detect 
design  and  material  differences  In  various  components.  The  second  plan 
makes  use  of  increased  severity  testing  to  reduce  the  required  sampie  sizes 
to  allowable  limits. 


For  Component  Tostlnc 


Before  testing  the  overall  fuze  performance  for  functioning  and 
safety,  it  Is  mandatory  that  each  of  the  major  components  of  the  fuze  be  qual¬ 
ity  controlled  to  tnsure  against  defective  material,  Also,  in  the  event  of  the 
possible  application  of  alternate  materials  for  particular  components,  each  of 
the  alternate  materials  should  be  subjected  to  test  in  order  to  Insure  that  the 
bust  material  (the  one  which  yields  tho  highest  degree  of  functioning  reliability 
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or 'accuracy)  is  selected.  A  faetofiolly  deBigned  experiment  incorporating  the 
various  alternate  materials  together  with  the  various  environmental  treatments 
can  be  set  up.  Test  results  would  indicate  the  particular  environmental  treat¬ 
ment  or  treatments  which  chow  significant  failure  rate  due  to  the  effect  of 
such  treatments.  Also  obtainable  therefrom  will  be  the  failure  rate  for  each 
of  uhe  alternate  types  of  materials  for  the  component.  After  these  are  deter¬ 
mined,  engineering  Judgment  can  be  invoked  to  determine  causes  for  failure 
and  modifications  can.  1*  made  to  remedy  such  causes.  Also,  the  material 
yielding  the  lowest  failure  rata  can  be  isolated  and  further  design  effort  con¬ 
centrated  on  that  type  of  material  which  yields  the  highest  degree  of  function¬ 
ing  reliability.  After  the  major  components  have  been  pre-tested  and  "bugs" 
withdrawn,  they  can  be  Incorporated  into  the  overall  fuze  design. 

b.  For  Complete  Frizes 

Fuzes  will  be  subjected  to  a  series  of  environmental  treatments 
In  seqtience  in  a  manner  simulating  that  in  which  they  are  normally. expected' 
to  occur.  Increased  severity  levels  of  these  environmental  treatments  will 
be  selected  on- the  .basis  of  engineering  judgment  as  being  those  which  appear 
to  be  mo3t  likely  to  cause*  failures  during  use.  The  statistical  test  plan  en¬ 
compasses  fractional  factorial  designed  experiments  which  would  subject  a 
minimum  of  fuze  samples  to  environmental  treatments  in  sequence.  Each 
environmental  treatment  would  consist  of  two  levels,  the  absence  of  the  par¬ 
ticular  treatment  and  the  presence  of  the  particular  treatment.  Moreover,  two 
types  of  response  data  could  be  elicited  therefrom.  One,  namely,  attribute 
data  and  the  other,  variable  data  which  measures  a  continuous  function  like 
arming  time,  self-destruction  time,  sustalner  switch  functioning  time*  etc. 
The  statistical  tost  plan  philosophy  provides  for  the  deliberate  inducing  of 
Increased  severity  levels*  of  each  of  the  environmental  treatments.  These  *• 
levels  will  Include  the  following: 


High  Level  -  -  —  - - Where  a  high  proportion  of  failures  Is 

expected. 

Intermediate  Level - .  -  -  Where  a  moderate  proportion  of  failures 

is  expected. 

Low  Level - - - Where  a  small  proportion  of  failures  Is 

expected. 


*  The  selection  of  levels  will  be  based  on  engineering  judgment. 
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The  test  plan  will  generate  Information  of  the  failure  rate  in  the  absence  and 
in  the  presence  of  a  particular  environmental  treatment.  Also,  it  will  produce 
variable  type  response  data  in  the  absence  and  in  the  presence  of  a  particular 
environmental  treatment  tvhich  later  can  be  translated  to  probability  of  success¬ 
ful  functioning.  Failure  rate  distribution  curves  will  be  obtainable  for  in¬ 
dividual  environmental  treatments.  These  curves  will  show  failure  rate  as  a 
function  of  level  of  severity  of  an  individual  environmental  treatment  and  of 
multiple  environmental  treatments.  The  following  types  of  curves  will  be 
obtainable: 


FAILURE  RATE  AS  A  FUNCTION  OT  LEVEL  OF  ENVIRONMENTAL  TREATMENT 


{ATTRIBUTE  DATAJ 


DESIGN  L  I 

LEVEL 


LEVEL  OF  ENVIRONMENT 


* 
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VARIABLE  MEASUREMENT  AS  A  FUNCTION  OF  ENVIRONMENTAL  TREATMENT 


z 

u 

2 


£  3 

x  2 
hi 
2 


LEVEL 

LEVEL  OF  ENVIRONMENT 


L  «LOV»  LEVEL 
I  »  INTERMEDIATE  LEVEL 
H  «  HIGH  LEVEL 


T  UPPER  CONFIDENCE  LIMIT 
i  AVERA6E 

LOWER  CONFIDENCE  LIMIT 


Curve  I  will  be  obtainable  for  each  of  the  environmental  treatments 
and  2 -factor  interactions  for  the  attribute  response. 

Curve  II  will  be  obtainable  for  each  of  the  environmental  treatments 
and  2 -factor  interactions  for  each  of  variable  responses.  Also,  it  is  ex¬ 
pected  that  Curve  n  can  be  translated  to  a  curve  of  probability  of  success¬ 
ful  functioning  as  a  function  of  level  of  environmental  treatment. 

The  aforementioned  plan  is  presented  as  an  alternate  approach  to 
testing  a  prohibitively  large  number  of  samples  at  the  design  level  in  order 
to  insure  reliable  functioning.  This  approach  proposes  to  accomplish  the 
same  objective  with  a  small  number  of  samples  by  obtaining  failure  rate 
distributions  over  the  range  of  increased  severity  levels  for  each  of  the 
environmental  treatments  imposed  upon  the  fuzes.  Also,  actual  measure¬ 
ment  data  of  specific  fuze  functions  will  be  obtainable  over  the  range  of 


Best  Available  Copy 


V 


344 


Design  of  Experiments 


increased  severity  levels  of  each  of  the  environmental  treatments.  These 
data  can  then  be  translated  to  probability  of  functioning.  For  example.  If  * 
we  define  the  probability  of  a  successful  function  as  the  probability  of  the 
continuous  variable,  e.g.,  arming  time,  being  greater  than  a  given  critical 
arming  time  or  lying  within  a  given  acceptable  region,  the  probability  of 
success  or  the  reliability  can  be  computed  from  Curve  II  data.  Thus  the 
probability  of  arming  time  being  greater  than  a  minimum  arming  time,  t 
critical,  is  as  follows:  - 


oo 


p  (t 


•  / 

—  ^critical)  “ 


f(t)  dt 


'critical  * 


ARMING  TIME  ,  t 


It  should  be  noted  that  the  aforementioned  proposed  program  represents 
the  approach  currently  being  implemented  since  it  appears  to  be  the  best 
approach  to  date  to  our  reliability  problem.  Although  the  program  does  not 
encompass  the  correlation  aspects  for  the  case  of  multiple  responses,  it 
is  contemplated  that  such  aspects  are  entirely  possible.  However,  since 
co-relationship  may  exist  among  several  attribute  responses,  among  several 
variable  responses,  and  also  among  attribute  and  variable  responses,  the 
manner  In  which  these  multiple  responses  can  be  handled  still  remains  to 
be  investigated.  . 
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